THE FUNDAMENTAL LEMMA OF JACQUET-RALLIS IN 
POSITIVE CHARACTERISTICS 



ZHIWEI YUN 



Abstract. We prove both the group version and the Lie algebra version of the 
Fundamental Lemma appearing in a relative trace formula of Jacquet-Rallis 
in the function field case when the characteristic is greater than the rank of 
the relevant groups. 



1. Introduction 

1.1. The conjecture of Jacquet-Rallis and its variant. In the paper [9], 
Jacquet and Rallis proposed an approach to the Gross-Prosad conjecture for uni- 
tary groups using relative trace formula. In establishing the relative trace formula, 
they needed a form of the Fundamental Lemma comparing the orbital integrals of 
the standard test functions on the symmetric space GL n (E)/ GL n (F) and on the 
unitary group U n (F), here E/F is an unramified extension of the local field F with 
odd residue characteristic. They explicitly stated (up to sign) a Lie algebra version 
of this Fundamental Lemma as a conjecture and verified it for n < 3 by explicit 
computation. Following this idea, Wei Zhang [13] stated the Lie group version of 
this Fundamental Lemma as a conjecture and verified it for n < 3. 

Let a be the Galois involution of E fixing F. Let f] E / F : F x — > {±1} be the 
quadratic character associated to the extension E/F. Let S n (F) be the subset of 
GL n (E) consisting of A such that Aa(A) — 1; let U n (F) be the unitary group 
associated with the Hermitian space E n with trivial disciminant. We also need 
the Lie algebra counterparts of the above spaces. Let s n (F) be the set of n-by-ra 
matrices with entries in E~ (purely imaginary elements in the quadratic extension 
E/F); let u n (F) be the set of n-by-n skew- Hermitian matrices with entries in E (the 
Hermitian form on E n has trivial discriminant). For a subset of K C gl n (E), let 
Ik deonte the characteristic function of K. The two versions of the Jacquet-Rallis 
conjecture arc the following identities of orbital integrals: 

1.1.1. Conjecture. 

(1) (Jacquet-Rallis, [9],) For strongly regular semisimple elements (see Defini- 
tion ] 2. 2. 1]) A £ s n (F) and A' e u„(F) that match each other (see Definition 
\2.5.1)) . we have: 

(1-1-1) of-^(l Sn{OF) ) = (-ir^O v A ?-^l UniOF) ). 
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(2) (Wei Zhang) For strongly regular semisimple elements A G S n (F) and A' G 
XJ n {F) that match each other (same notion as above), we have: 

(1.1.2) of —"(i s „ (0f) ) = (-lr^o^r^iu,,^)). 

Here, for elements A, A' G Qi n {E), and smooth compactly supported functions 
f,fon S l n (E), 

gl„_ 1 ,, (/) = f f( g -iAg)r, E/F (det(g))dg 

JGL„_i(F) 

O^tf') = / f'{jg- 1 A'g)dg 

JVn-l(F) 

For the definition ofv(A), see Definition \2.2.2[ 

In the above two situations, if A does not match any A' , then the LHS of (jl.l.ip 
or (|1.1.2[) is zero. 

Let H n (F) be the set of Hermitian matrices in GL n (E) and \) n (F) be the set 
of n-by-n Hermitian matrices in Ql n (E) with respect to the chosen Hermitian form 
with triival disciminant. We have the following variant of the Conjecture |TTTTT| 

1.1.2. Conjecture. 

(1) For strongly regular semisimple elements A € gl n (F) and A' 6 t)n(F) that 
match each other (in the similar sense as Definition \2.5.l\) . we have 

(i-i-s) of — f '(i BUOF) ) = (-lr^o^- 1 ^^)). 

(2) For strongly regular semisimple elements A £ GL„(i ? ) and A' G H n (F) 
that match each other, we have 

(i-i-4) of—"(i GL „ (OF) ) = (-ir^oSr 1 (iH B ( OF) ). 

(3) In the above two situations, if A does not match any A' , then the LHS of 
(fTX3| or (flX4l) is zero. 

I. 2. Main results. The main purpose of the paper is to prove the above Conjec- 
tures in the case F is a local function field (i.e., of the form k((w)) for some finite 
field k) and char(i^) > n (see Corollary [27772]) . 

We first do some reductions. In fact, as observed by Xinyi Yuan, (|1.1.3|) is sim- 
ply equivalent to Conjecture (jl.l.ip because multiplication by a purely imaginary 
element in E~\0 interchanges the situation. In Proposition 12.6. we show that 
the group version (|1.1.2jl follows from the Lie algebra version (jl.l.ip for any F 
(of any characteristic); the same argument shows that (|1.1.4p follows from (jl.l.3p . 
Therefore, the orbital integral identity (jl.l.ip for Lie algebras implies all the other 
identities, for any F. Moreover, the vanishing result in Conjecture 11.1.1( 3) and 

II. 1.2( 3) follows from a cancellation argument (see Lemma [2373]) . 

To prove (jl.l.ip in the case char(F) > n, we follow the strategy of the proof 
of the Langlands-Shelstad Fundamental Lemma in the Lie algebra and function 
field case, which is recently finished by Ngo Bao Chau ([Hj), building on the work 
of many mathematicians over the past thirty years. The geometry involved in 
the Langlands-Shelstad Fundamental Lemma consists of a local part — the affinc 
Springer fibers (cf. |6j) and a global part — the Hitchin fibration (cf. Ngo [11], [P2]). 
Roughly speaking, the motives of the affine Springer fibers, after taking Frobenius 
traces, give the orbital integrals; the motives of the Hitchin fibers can be written as 
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a product of the motives of affine Springer fibers. The advantage of passing from 
local to global is that we can control the "bad" (non-computable) orbital integrals 
by "nice" (computable) orbital integrals, using global topological machinery such 
as perverse sheaves. 

In the following two subsections, we reformulate Conjecture II. l.lf 1) using local 
and global moduli spaces, and indicate the main ideas of the proof. 

1.3. The local reformulation. As the first step towards a local reformulation, we 
translate the problem of computing orbital integrals into that of counting lattices. 
In [9], the authors introduced (2n — l)-invariants associated to an element A G 
Ql n (E) with respect to the conjugation action of GL n -i{E). Here the embedding 
GL n -i(E) <^-> GL n (E) is given by a splitting of the vector space E n = E n ~ x E. 
The splitting gives a distinguished vector eo (which spans the one dimension direct 
summand) and a distinguished covector (the projection to Eeo). In this paper, 
we use a different (but equivalent) set of invariants a = (ai,-- - , a n ) and b — 
(bo, ■ ■ ■ , 6 n _i) for A: the a,'s are the coefficients of the characteristic polynomial 
of A and bi = e^A^o (so that b = 1). We say that A 6 S n (F) and A' <G u n (F) 
match each other if they have the same collection of invariants viewed as elements 
in Ql n {E). 

We fix a collection of invariants (a, b) which is integral and strongly regular 
semisimple (see Definition 12.2.11 and Lemma 12.2. 4|) . Then we can associate a finite 
flat Op-algebra R a (see tJ2.2[) . The invariants b gives an i? a -linear embedding 7^5 : 
R a^ Ra = Hornet (R a , O f ). We can rewrite the LHS of (TITTTTj) as 

0f"-^(l Sn{0F) ) =±X)(-1) < #M£ 6 

i 

where each M\°£ b is the set of i? a -lattices A such that JJ„cAc and leng OF (R^ : 
A) = i (see Notation [TXT]) . 

The E- vector space R a (E) — R a ®o F E carries a natural Hermitian form given 
also by b. Similarly, we can rewrite the RHS of (jl.l.ip as 

0^- 1 (l u „ (OF )) = #iV^ 

where iV^°£ is the set of i? a (C£;)-lattices A', self-dual under the Hermitian form, 
and R a {0 E ) C A' c <(O b ). 

When F is a function field with residue field k, there are obvious moduli spaces 
of lattices M l °^ b and J\f^ defined over k such that M^ b and N l °% are the set of 
fc-points of M\°ab an( i -Nab- By Lefschetz trace formula for schemes over k, the 
Conjecture [T7T7TJ is a consequence of the following theorem, which is the main local 
result of the paper 

Main Theorem (Local Part). Suppose char(F) = char(fc) > max{n, 2} and 
T)E/F{A a ,b) = 1- Then there is an isomorphism of graded Frobk-modules: 

val F (A a , b ) 
i=0 

For more details on the notation, see ^2.71 This theorem is deduced from the 
global Main Theorem, which we shall give an overview in the next subsection. 

We point out that there are several easy cases of the Conjecture II .1.11 which are 
verified in §2.51 without any restriction on F. 
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1.4. The global approach. The global geometry related to the Jacquet-Rallis 
Fundamental Lemma is the modified versions of Hitchin moduli spaces for the 
groups GL„ and U n . The geometry of Hitchin fibrations is studied in great detail 
by Ngo [12] and Laumon-Ngo [10] . the latter treats the unitary group case and is 
especially important for the purpose of this paper. 

We fix a smooth projective and geometrically connected curve X over k; to study 
the unitary group, we also fix an etale double cover X' — ► X. We fix two effect 
divisors D and Do on X of large enough degrees. 

We introduce the stack A4, classifying quadruples (£, <fi, A, jj) where £ is a vector 
bundle of rank n; 4> is a Higgs field on £ ; A : Ox (—Do) — ► £ is the global counterpart 
of the distinguished vector eo and fi : £ — » Ox (Do) the global counterpart of the 
distinguished covector eg. The stack M. is the disjoint union of M.i (i 6 Z), 
according to the degree of £ . 

We also introduce the stack Af, classifying quadruples (£' , h, <j>' , //) where £' is 
a vector bundle of rank n on X'; h is a Hermitian structure on £'• $ is a skew- 
Hermitian Higgs field on £' and // : £ — > Ox'(-Do) is the distinguished covector 
(the distinguished vector is determined by // using the Hermitian structure). 

Both Aii and A/" fiber over the "Hitchin base" AxB, classifying global invariants 
(a,b). In [J3J we prove the following geometric properties of these moduli spaces 

• (Proposition E2H and [3X21) Over the locus „4 int x B x , A4j nt (for "non- 
extremal" i) and Af mt are smooth schemes over k and the "Hitchin maps" 

jint . M mt _^ ^int x £X and g int . tfmt _^ ^int x &re p roper; 

• (Proposition 13.5.2]) Fix a Serre invariant 6 > 1 (see §3.5|) . for deg(D) and 
deg(Do) large enough, the restrictions of /? nt and <7 mt to A- 5 x S x are 
smaH (cf. Notation II. 6. 6j) : 

• fProposition l3.4.11 and [3TT!2^) The fibers M.i^ a ,b and M a ,b can be written as 
products of local moduli spaces Aif a b and A/J defined in a similar 
way as A4^ 6 and A/]° b c . 

The global part of the Main Theorem is 

Main Theorem (Global Part). Fix 8 > l 7 deg(D) > c s and n(deg(L> ) -.9 + 1) > 
5 + gy ■ Then there is a natural isomorphism in D h c (A- s x B x ): 
d 

(1-4.1) (J) f"^L d -i\ A <5 Xt3 x = .g* nt Q£U<^x8x- 

i— — 

Here d = n(n — l)deg(£>)/2 + ndeg(_D ); is a local system of rank one 

and order two on A4J nt (see tj4.1|) , which is a geometric analogue of the factor 
rj E / F (det(g)) appearing in the orbital integral Q^ 11 - 1 ' 71 (l Sn (o p -\). 

The smallness of f\ nt and g lnt over A- s x B x and the smoothness of M. - nt and 
A/" mt are the essential geometric properties that enable one to prove the above 
theorem. In fact, these two properties imply that both sides are middle extensions 
of some local system on some dense open subset of A- 5 x B x , and to verify (|1.4.1|) . 
we only need to restrict to a nice open dense subset where both sides are explicitly 
computable. 

However, there is one technical difficulty. The schemes M.™* are visibly smooth 
only for "non-extremal" i's (see Proposition 13.2.61 for the range of i where A4\ nt is 
proved to be smooth), therefore the above argument only works for these i's. To 
get around this difficulty, we enlarge D and compare the moduli spaces defined 
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by two different Do's. Indices i which are extremal for the original Do become 
non-extremal for the new Do- Details are given in £ 14.41 

Finally, we use the global part of the Main Theorem to prove the local part. 
We identify F with the local field associated to a fc-point xo on the curve X. For 
local invariants (a , 6°), we may choose global invariants (a, b) such that the global 
moduli spaces Aii t a,b and 7V a ,b, when expressed as product of local moduli spaces, 
are very simple away from xq- Taking the Frobenius traces of the two sides of 
(11.4. ip and using the product formulae, we get a formula of the following form 

Tt(ftobfc,M 1B0 ). J] Tr(Frob fe ,M a ) = Tr(Frob fe ,Ar xo )- J] Tr(Frob fc , N x ) 

x^xo X^XQ 

where M x and N x are the cohomology groups of the relevant local moduli spaces. It 
is easy to show Tr(Frobfc, M x ) = Tr(Frobfe, N x ) for all x ^ xo, but to conclude that 
Tr(Frobfc, M Xo ) — Tr(Frob&, N XQ ), which is what we need, it remains to make sure 
that the terms Tr(Frobfc, N x ) are nonzero for all x ^ xq. This is another technical 
difficulty of the paper, and is responsible for the length of £15.21 

1.5. Plan of the paper. In [21 we first fix notations and introduce the invariants 
(a, b). We then reformulate the problem into counting of lattices in £12.31 and £12.41 
We verify a few simple cases of Conjecture II. 1.11 in £ 12.51 including Conjecture 
11.1.1( 3). We deduce the group version Conjecture 11.1.1( 2) from the Lie algebra 
version Conjecture 11.1.1( 1) in §2.61 Except for £12. 7\ we work with no restrictions 
on F. 

In £j3j we introduce the global moduli spaces Aii and Af and study their geometric 
properties. 

In £JU we formulate and prove the global part of the Main Theorem. To this end, 
we need to study perverse sheaves on the symmetric powers of curves, especially the 
"binomial expansion" formula (Lemma l4.2.3|) . which demystifies the decomposition 
(fTXTjl . 

In £j5l we deduce the local part of the Main Theorem from the global part. 

1.6. Notations. 

1.6.1. Let O be a commutative ring. For a scheme X over O or an O-module M, 
and an O-algebra R, we let X(R) be the i?-points of X and M(R) := M ®o R- 

For a DVR O and two full rank O-lattice Ai, A2 in some Frac(0)-vector space 
V, we define the relative length leng (Ai : A2) to be 

leng (Ai : A 2 ) := leng (Ai/Ai n A 2 ) - leng (A 2 /Ai n A 2 ) 

where leng (— ) denotes the usual length of a torsion O-module. 

1.6.2. Coherent sheaves are denoted by the calligraphic letters £ ,JF, £, • • • ; con- 
structible Q^-complexes are denoted by the capital letters L,K, - ■■ . 

1.6.3. From §3, we will work with a fixed smooth base curve X over a field k. For 
a morphism p : Y —> X and a closed point x S X, we denote by Oy.x the completed 
semilocal ring of Oy along p^ 1 (x). 

If Y is a Gorenstein curve, let ujy/x be the relative dualizing sheaf 

UJy/X = uy/k ®P* w x/fe- 
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For a coherent sheaf T on Y, let JF V = Hom y (T ', u>y/ x ) be the (underived) 
Grothcndieck-Serre dual. When we work over an extra parameter scheme S so 
that p : Y — > X x S, then v means Hom y (— , lu y /xxs)- 

1.6.4. For an etale double cover (a finite etale map of degree 2) tt : X' — > X of 
a scheme X, we decompose the sheaf 7r*Q f into ±l-eigenspaces under the natural 
action of Z/2 = Aut(X'/X): 

The the rank one local system L satisfies L® 2 = Q e . We call L the local system 
associated to the etale double cover ir. 

1.6.5. We use the terminology "middle extension" in a non-strict way. If K is a 
Q^-complex on a scheme X , we say X is a middle extension on X if for some (and 
hence any) smooth open dense subset j : U c — > X over which if is a local system 
placed at degree n, we have 

K = jw(j*K[n + dim X]) [-n - dimX]. 

1.6.6. Recall from [7, §6.2] that a proper surjective morphism / : Y — » X between 
irreducible schemes over an algebraically closed field f2 is called small if for any 
r > 1, we have 

(1.6.1) codim x {a; G X|dim/ _1 (a:) > r} > 2r + 1. 

We will use this terminology in a loose way: we will not require Y to be irreducible 
(but we do require all other conditions). This is just for notational convenience. 
The main property that we will use about small morphisms is: 

IfY/Cl is smooth and equidimensional, then f*Qg is a middle extension on X. 

1.6.7. For a finite field k, we denote by Frob^ the geometric Frobenius element in 
Gal(fc/fc). 

Acknowledgment. The author would like to thank Xinyi Yuan and Wei Zhang 
for suggesting the problem and for helpful discussions. He also thanks Joseph 
Bernstein, Julia Gordon, Yifeng Liu and Shou-Wu Zhang for useful suggestions. 
Different parts of this work were conceived during the workshop on Trace Formula 
held in Columbia University in Dec. 2008, and the AIM workshop on Relative 
Trace formula and Periods of Automorphic Forms in Aug. 2009. The author would 
like to thank the organizers of these workshops. 

2. Local formulation 

2.1. The setting. Let F be a non- Archimedean local field with valuation ring 
Of, uniformizing parameter vj and residue field k = ¥ q such that q is odd. Let 
E be either the unramified quadratic extension of F (in which case we call E/F 
is nonsplit) or E = F x F (in which case we call E/F is split). Let Oe be the 
valuation ring of E and k' be the residue algebra of k. Let a be the generator of 
G&\(E/F). Let t] E / f ■ F x /~NmE x — * {±1} be the quadratic character associated 
to the extension E/F: this is trivial if and only if E/F is split. We decompose E 
and Oe according into eigenspaces of a: 

E = F ® E~; O e = O f ®O e 

where a acts on E~ and O e by —1. 
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We fix a free O^-modulc W of rank (n — 1). Let V = W © Of • eo be a free 
0F-modulc of rank n with a distinguished element eo and let eg : V — > Of be 
the projection along W such that eg(eo) = 1. Let W v — Homo F (W, Of) and 
F v =Hom OF (y,0 F ). 

Let GL„_i = GL(VF) and GL„ = GL(V) be the general linear groups over Of- 
We have the natural embedding GL„_i GL„ as block-diagonal matrices: 

A 

1 

Let gi n be the Lie algebra (over Of) of GL„ consisting of Op-linear operators 
on V. Let 

s n (0 F ) := {A g flt„(0 B )|A + (7(A) = 0}. 
Then s„(F) is be the subset of gl n (E) consisting of matrices with entries in E~ . 
The group GL„ acts on s n by conjugation. 

Let (•,•) be a Hermitian form on W with trivial discriminant and extend this 
to a Hermitian form on V by requiring that (W, eo) = and (e ,e ) = 1. These 
Hermitian forms define unitary groups U„_i = U(W, (•, •)) and U„ = U(V, (-, •)) 
over Of- We also have the natural embedding U„_i ^ U„ as block-diagonal 
matrices: 

A 

1 

Let u n be the Lie algebra (over Of) of U„, i.e., 

u n (0 F ) = {A e l n (O E )\A + A* = 0} 

where A* is the adjoint of A under the Hermitian form (-, •). Then U n acts on u n 
by conjugation. 

2.2. The invariants. For any A e gl n (E), and i = 1, • • • , n, let 

:=Tr(/\A) 

be the coefficients of the characteristic polynomial of A, i.e., 

n 

det(iid y -a) = t n + ^T{-iy ai {A)t n -\ 

1=1 

For i = 0, • • ■ , n — 1, let 

& i (^)=eS(^ i e ). 

so that 6o = 1- The 2n-tuplc (a (A), 6(A)) of elements in E are called the invariants 
of A £ gl n (E). It is easy to check that these are invariants of gl n (E) under the 
conjugation action by GL„_i(£'). 

2.2.1. Definition. An element A G gl n (E) is said to be strongly regular semisimple 
with respect to the Gh n -i(E)-action, if 

(1) A is regular semisimple as an element of gl n (E); 

(2) the vectors {eo, Aeo, ■ • ■ , A n ~ 1 e } form an .E-basis of V(E); 

(3) the vectors {eg, e A, • • • , e^A™ -1 } form an E-basis of V V (E). 

2.2.2. Definition. For A e gl n (E), define v(A) E Z to be the F- valuation of the 
n-by-n matrix formed by the row vectors {eg, e$A, • • • , egA™ -1 } under an Op-basis 
of\> v . 
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For a collection of invariants (a, b) £ E 2n (we allow general bo £ E, not just 1), 
we introduce a finite i?-algebra 

R a (E) := £[t]/(t B - ax^- 1 + ■■■ + (-l) n a n ) 

Let Rq(E) := Hom E (R a (E), E) be its linear dual, which is naturally an R a (E)- 
module. The data of b gives the following element b' £ R^(E): 

(2.2.1) b':R a (E) -» £ 

i* i-» bi, i = 0, 1, • • • ,n - 1. 

which induces an i? a (i?)-linear homomorphism 

7 ; b :i? a (£)^i?r(i?). 
In other words, 7^ h is given by the pairing: 

(2.2.2) R a (E)® E R a {E) -> £ 

(x,y) 1 * 

2.2.3. Definition. The A-invariant A aj f, of the collection (a, 6) G E 2n is the de- 
terminant of the map 7^ b under the i?-basis {l,t, ••• , i™ -1 } of R a (E) and the 
corresponding dual basis of i?^ . 

It is easy to see that for A e gl n (E) with invariants (a, 6), A a 5 is the determinant 
of the matrix (eoA l+:, eo)o<ij<ri- 

2.2.4. Lemma. Let A £ g{ n (E) with invariants (a,b) £ E 2n . Then A is strongly 
regular semisimple if and only if 

(1) R a (E) is an etale algebra over E; 

(2) A 0) 6 ^ 0; i.e., 7^ b : R a (E) — > R^(E) is an isomorphism. 

Proof. Condition (1) is equivalent to that A is regular semisimple as an element of 
gl n (E). Condition (2) is equivalent to that the matrix (eQA 1+J eo)o<ij<n is nondc- 
generate. Since this matrix is the product of the two matrices (eg, eg A, • • ■ , eJjA" -1 ) 
and (eo, Aeo, ■ ■ ■ , A n ~ 1 eo), the nondegeneracy of (e^A^i eo)o<i,j<n is equivalent to 
the nondegeneracy of (eg, e^A, ■ ■ ■ , e^A 71 ' 1 ) and (eo, Aeo, ■•• , A n ~ 1 eo), which are 
the last two conditions of Definition 12.2.11 □ 

2.2.5. Remark. From the above Lemma, we see that the strong regular semisim- 
plicity of A £ Ql n (E) is in fact a property of its invariants (a, b). Therefore, we 
call a collection of invariants (a, b) strongly regular semisimple, if it satisfies the 
conditions in Lemma [2.2.41 

If the invariants (a, b) are elements in O e , we get a canonical Cg-form R a [Q E ) 
of R a (E) by setting 

(2.2.3) R a (0 E ) := E [t]/(t n - a^ 1 + ■■■ + (-l) n a„) 

Let Ra(0 E ) = Uom E (R a (0 E ),0 E ) be its dual. Then 7^ restricts to an R a (0 E )- 
linear map 

y S|6 R a (o E ) - n»(o E ). 

For A £ 5 n (F) or u n (F) with invariants (a, b), it is obvious that a*, 6j £ i?""^ 1 ) . 
Suppose further more that a%, bi £ O e ~^ . We extend the involution a on O e to 
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an involution or on R a (C>E) by requiring gr{£) = —t. The involution or defines 
an Op-form of R cl {Oe)'- 

R a := R a {0 E Y R 

Let :— Homo F (R a , Of)- The map 7^ b restricts to an i? a -linear homomorphism 
(2.2.4) 7a , b : R a -» <. 

and 7 a b = 7a. 6 8>o F Cs- Since i? a is an Op-form of R a {C>E) defined before, there 
is no confusion in using the notations i? Q (Oe) or R a (E) (cf. Notation ll.G.ip . 

2.3. Orbital integrals for s n (F). Let A G s n (F) be strongly regular semisimple 
with invariants (a, b). Let 

f°" I ' , '( 1 »»(Oj-)) : = / U n (o F )(9~ 1 ^9)VE/F(<iet(g))dg 

JGL„-i(F) 

where is the Haar measure on s n (F ) such that vol(s„(C>p), <i</) = 1 and l Sn (e> F ) 
is the characteristic function of s„(Op) C s n (F). 

2.3.1. Remark. It is easy to see if the orbital integral 0^ L " -1: ''(l 5ii (e> F )) ^ 0, then 

ai M g 0^ ir . 

Now we suppose (a, 6) is strongly regular semisimple and a,, 6, G O^ -1 ^ . We 
view R a as a sublattice of R^ via the map 7 a , ■ R a ^ Ra- Then 

leng OF (i?^ : i? a ) = val F (A Q:b ). 

For each integer < i < valp(A Q)0 ), let 

M l°a,b ■= {i?a-lattices A\R a C A C fl a v and leng OF (i?^ : A) = i} 

2.3.2. Proposition. Let A G S n (F) be strongly regular semisimple with invariants 
a,i,bi G O e ~^ ^ , then 

val F (A a .i,) 

of *- uv Q., nlOF) ) = ve/f^) v{a) £ VE/F (my#M^ b . 

i=0 

Proof. By Definition 12.2.1 l[2l) . we have a c-equivariant _E-linear isomorphism: 

(2.3.1) i':R a {E) ^ V(E) 

(2.3.2) f h-> A 4 e . 
Therefore t' restrict to an _F-linear isomorphism 

l : R a (F) ^ V(F). 

Define R a ,w ■= Ra n ^(W^F)) and : = i?v n L - 1 (W(F)). We define some 

auxiliary sets 

Xi, A := { 5 eGL„_ 1 ( J F 1 )/GL„_ 1 (O i ,)|. g - 1 AgGB„(0 F ),val F (det( 3 ))-z}; 
:= { Op-lattices L c W(F)|A(L) c C^L, len gc , F (L : = i}; 
M If a ,b : = {Op-lattices A w c i? Q:lv (F)|AvK © Opl/? C R a (F) is stable under R a , 
leng 0p (R% w : K w ) = i}. 

where 1r G R a is the identity element. 
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Note that the group GL n ~i(F) acts transitively on the set of 0_F-lattices in 
W(F) by left translation, and the stabilizer of W equal to GL„_i(0_f )■ Therefore 
we get a bijection 

g i > gW. 

We identify R a e — > R% both as OF-lattices in V(E) via l. Observe that 
leng OF «w : W) = leng 0jr (^ : V) = v(A), 
therefore we have a bijection 

1U v(A)-i,a,b A M 

h-w t(Av^). 

Finally we have a bijection 

J/W r A/f loc 

N1 i,a.b ^ 1V1 i,a,b 

A w i-> Aw/eOFlfl, 
We check this is a bijection. On one hand, for Aw G ^ & , we have 
# a C R a (A w © OfIh) c A w © Of 

We also have 

b'{R a (A w © O f 1k)) C &'(A w © O f 1r) C Of, 
therefore A w ®O f 1 r <zR^. This verifies A w © O f 1 r G M'° c i6 . 

On the other hand, we have to make sure that every A £ Af^ c fe has the form 
A = Aw ® Opln for some lattice A\y c i? a ,w(-F)j i-e., Of If C A is saturated. 
But we can factorize the identity map on Of as 

Q F R a 1* i?v q f . 

Therefore for R a C A C i?^ , O f 1r C A is always saturated. 

Now that we have set up a bijection between Xi,a and M y ° l ^_ i a b , we have 

= 77f/f(^)" (A) E ^f/fM'#M^> 

i 

□ 

2.4. Orbital integrals for u n (F). 

2.4.1. Remark. It is easy to see if the orbital integral O^" -1 (l Sn (Op)) ^ 0, then 
a, A G O^ -15 '. 

Now we suppose (a, 6) is strongly regular semisimple and ctj, &i G O^ - ' 1 . We 
identify R a (0 F ) as a sublattice of R^(Oe) via 7^ b . Recall from (|2.3.1| that we 
have an isomorphism il : R a (E) — » ^(E 1 ). The transport of the Hermitian form 
(-, ■) to R a {E) via 1! is given by: 

(2.4.1) (x,y) R = b'(xa R (y)). 

where b' : i? a (#) -> E is defined in (|2~2~Tj) . 
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2.4.2. Remark. Since the Hermitian form (•, •) has trivial discriminant, so is (•, -)r. 
Therefore if A' £ u n (F) has invariants (a,b), then ri E /p(A a ,b) = 1. 

Recall that for an O^-lattice A' C R a {E), the dual lattice under the Hermitian 
form (•, -)-R is the ©^-lattice 

A' 1 - := {xeR a (E)\(x,A') R cO E }. 

Such a lattice is called self- dual (under the given Hermitian form) if A /J - = A'. 
Comparing with the pairing (|2.2.2|) . it is easy to see that R^(Oe) is the dual of 
R<i{Oe) under the Hermitian form (•, We define 

N l °l := {self-dual R a (0 E )-lattices k'\R a (0 E ) cA'c K(0 E )}. 

2.4.3. Proposition. Let A' £ u n (F) be strongly regular semisimple with invariants 
Qi,bi £ O e ~^ ^ , then 

o^r 1 (iu„(o P) ) = #<6 c - 

Proof. The argument is similar as the proof of Proposition 12.3.21 Let 

Y A , = {g e U n _ 1 (F)/U n _i(0 F )|<T 1 A , 9 e u n (0 F )}. 

Then U n _i(F) acts transitively on the set of self-dual O^-lattices in W(E), such 
that the stabilizer of W{Oe) is U„_i(C_f). Therefore we get a bijection 

Hence 

o^- 1 (i„„(o F) ) = #^ = 

□ 

2.5. The Fundamental Lemma and simple cases. 

2.5.1. Definition. Two strongly regular semisimple elements A £ s n (F) and A' £ 
u n (F) are said to match each other, if they have the same invariants. 

Now we have explained all the notions appearing in Conjecture [TTTTTJ By Propo- 
sition [23?2] and [2A3l the Conjecture ll.l.lf 1)(3) are implied by 

2.5.2. Conjecture. For any strongly regular semisimple collection of invariants 

(a,b) such that ai,bi £ O e ~^ (in particular we allow arbitrary bo £ Of), we 
have 

val F (A ai i,) 

(2-5.1) Yl VE/F^Y#M^ b = #< b c . 

i=0 

In the rest of this subsection, we prove some easy cases of the Conjecture 12.5.21 
by straight-forward counting argument. 

2.5.3. Lemma. The Coniecture \2.5.2\ is true if r] E i F(Aa,b) 1j * n which case both 
sides of \2.5.1\) are zero. In particular, Coniecture M.l. IV 3 ) holds. 
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Proof. The situation T)E/F(A a ,b) ^ 1 happens if and only if E/F is nonsplit and 
valF(A Qib ) = leng OF (it^ : R a ) is odd. In this case, leng 0js (R^{Oe) ■ Ro.{Oe)) is 
odd, therefore there are no self-dual lattices in R a (E), i.e., N^g = 0. 

Now we show that the LHS of l|2.5.ip is also zero. For an Op-lattice A £ R a (F), 
the linear dual A v = Homo F (A, Of) can be naturally viewed as another O^-lattice 
of R a (F) via the identification ^a.b ■ R a (F) ^ R^(F). It is easy to check that if A 
is stable under multiplication by i? a , the same is true for A v . This operation sets 
up a bijection 

I \ y ■ Af loc M loc 

\. I ■ MI i.a.b * - ,W val F (A a-i) )-j : a : b- 

Since E/F is nonsplit and valj?(A aj ;,) is odd, then by Proposition 12.3.21 

|val F (A .(,)/2j 

This completes the proof. □ 

2.5.4. Remark. In [5], Jacquet-Rallis showed that every strongly regular semisimple 
A' e u n (F) matches some A £ s n (F); conversely, a strongly regular semisimple 
A £ s n (F) matches some A' £ u n (F) if and only if r] E / F (A a _b) = 1, where (a, b) are 
the invariants of A. We have seen from Lemma 12.5.31 that if A £ s n (F) does not 
match any element in u n (F), then 0^ L " _1 ' r '(l 57l (Of)) = 0. This vanishing result 
was also conjectured in [9]. 

2.5.5. Lemma. The Coniecture \2.5.2\ is true if E/F is split. 

Proof. Fix an isomorphism E = F © F such that a interchanges the two factors. 
Using this, we can identify R a (E) is with R a (F) © R a (F) and the Hermitian form 
(•, -)r takes the form: 

(x ®y,x'® y') R = b(xy') © b(x'y) £ E, x, y, x', y' £ R a {F). 

Therefore each i? a (0F)dattice R a (0 E ) C A' C R^(Oe) has the form A' = Ai © A 2 , 
with R a C A,; C R^. The self-duality requirement is equivalent to A 2 = lali^-i ) 
(note that A^ C Ra{F) and recall the isomorphism 7 Q , b : R a (F) ^> R^(F)). In 
this way, we get a bijection 

val F (A a , 6 ) 

A i ► A© 7 - b 1 (A v ). 

Therefore, 

which verifies Conjecture 12.5.21 in the split case because tje/f is trivial. □ 

2.5.6. Lemma. The Conjecture 1 2. 5. 2] is true if R a is a product of DVRs. 

Proof. By reducing to the lattice-counting problems, it is clear that it suffices to deal 
with the case R a is a DVR. Since we already dealt with the split case, we may assume 
E/F is nonsplit. Let k(R a ) be the residue field of R a and vjr be a uniformizing 
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parameter of R a . Then R% = vj u d R a for some integer d = leng fia (i?^ : R a ). We 



have 

(2.5.2) val F (A a , h ) = leng OF (i?^ : R a ) = d[k{R a ) : k] 

We already solve the case when val F (^ a ,b) is odd in Lemma [2. 5. 31 and when E/F is 
split in Lemma l2.5.51 Now suppose vali?(A Qj b) is even and E/F is nonsplit. In this 
case, either d or [k(R a ) : k] has to be even. Let us explicitly count the cardinalities 
of Mfc b and N\° c b . 

On one hand, the only i? a -lattices which sit between R a and R^ are R a for 
< j < d, and \eng 0p (R^ : w R j R a ) = (d - j)[k{R a ) : k}. Therefore 

E, T\i-HAT loc i\{d-j)[k(R a ):k] _ ( d+1 [fc(i? a ) : fc] even; 

{ 1J #^i,o,6 - l > -\l [k(R a ) : jfc] odd. 

i j=0 

On the other hand, if [k(R a ) : k] is odd and d is even, R a {OE) remains a DVR, 
therefore m R d ^ 2 R a (OE) is the unique self-dual R a {OE) lattice between R a (OE) 
and R^(Oe) = ^ d Ra{0 E )- If [k{R a ) ■ k] is even, then we can identify R a (0 E ) = 
R a © R a such that an acts by interchanging the two factors. In this case 

consists of lattices w~j^ R a © + ^ R a for Q < j < d. In any case, we have 

i 

□ 

2.6. From the Lie algebra version to the group version. As mentioned in 
the Introduction, it is the group version identity (|1.1.2p which is directly relevant 
to Jacquet-Rallis's approach to the Gross-Prasad conjecture for the unitary groups. 
In this subsection we deduce the group version (|1.1.2|) from the Lie algebra version 
(ll.l.ip . The same argument also shows that (jl.l.4[) follows from (jl.l.3[) . 

For an element A s GL n (E), viewed as an element in gl n (E), the invariants 
a,i(A),bi(A) and v(A) are defined as in <j2.2l When a^bi 6 Oe, we introduce the 
0£-algebra 

(2.6.1) R a (0 E ) = OE^t- 1 ]/^ - ai t n - x + ■■■ + {-l) n a n ). 

We view Z' a = SpecR a (0£) as a subscheme of SpecO_E x G m which is finite flat 
over Spec0£ of degree n. Let 9 be the involution on SpecO^ x G m which is the 
product of a on Oe and 1 \— > t _1 on G m . The fixed point subcsheme under is the 
unitary group \Jq e /q f (1) under Spec Of. 

Recall that S n (0 F ) = {4e GL n (0 E )\Aa(A) = 1}. For an element A in either 
•Sri (Of) or U„(Of), the subscheme Z' a is stable under 0, hence determining a 
subscheme Z a C \Jq e /q f (1), finite flat of degree n over Spec Op. Let R be 
the coordinate ring of Z a , which is a finite flat 0F-algebra of rank n satisfying 
Ra ®e> F Oe = Ra(Cfi)- The invariants bi determines an R a -linear map j a< j, : 
R Q -> R^, as in (j2~2~4)l . 

2.6.1. Proposition. Conjecture \1J~J^ 2) follows from Conjecture 

Proof. Using the same argument for Propositions 12.3.21 and 12.4.31 we reduce the 
orbital integrals in (|1.1.2[) to counting of points in the corresponding sets b and 

N^°j, defined using R a instead of R a . Therefore, it suffices to find Si, bi 6 0^T (_1 ' ) 
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and an isomorphism of Of -algebras p : Ra — > R a such that the following diagram 
is commutative: 



(2.6.2) 



Ra 

i 

R 



Ta.i 



•R„ v 



Moreover, once we find p : Ra — ► R a , the choice of b is uniquely determined by the 
diagram (|2.6.2p . because the data of b and determine each other, as seen in 
(|2.2.f p and (]2.2.2[) . Therefore we only need to find a such that R a is isomorphic to 

Consider the special fiber of the Of -scheme Z a , which is a finite subscheme of 
Ufe' /fc(l) of degree n. Since U fc / is a smooth curve, any subscheme of it can be 
embedded into AjL In other words, there is a surjection of algebras k[s] -» R a ®o F fc. 
Lifting the image of the generator s to an element of R a , we get a surjection of 
Of -algebras Of [s] -» R a (surjectivity follows from Nakayama's lemma). In other 
words, Z a can be embedded as a subscheme of Spec Of x A 1 . It is well-known that 
any such finite flat Of -subscheme of Spec Of x A 1 is defined by one equation of 
the form t n — cii™ -1 + • • • + (— l) n c n for some Ci G Of, i.e., there is an isomorphism 



(2.6.3) 



F [s}/(t n -c 1 t n - 1 + --- + (-l) n c n ) 



R„ 



Let j G O^, n O e be a purely imaginary unit element and let = j % q G O 
then we have an isomorphism Ra ^ R^ by sending t i— > j _1 s. Composing with 
(I2.6.3p . we get the desired isomorphism of Of -algebras p : Ra — » R Q - This com- 
pletes the proof. □ 

2.7. Geometric reformulation. In this subsection, we assume char(F) = char(fc). 
In this case, we can interpret the sets M\°£ b and N a ° b as /c-points of certain schemes. 

Fix a strongly regular semisimple pair (a, b) such that a,, 6j G Of - *' 1 . For 
< i < valf (A a ;,), consider the following functor 



Ra ®k S - 

modules A 7 



Ra\ 

R„ 



h Os C A C M k Os, 

3fc Os/A is a vector bundle of rank i over S 

loc 



It is clear that this functor is represented by a projective scheme -A4°^ b over fc, and 



M. 



loc 
i.a.b 



= M 



Similarly, we have a projective scheme Af}?§ over k representing the functor 



S 



Self-dual R a (O x )£ 
Os-modules A' 



We also have 



R a {0 E ) 

Nh = 



h O s cA'cR a , (0 E )M k Os, 

3fc O5/A' is a vector bundle over Os 



Let £ be a prime number different from char(fc). Let Q^(%'/fe) be the rank one 
Qf-local system on Specfc associated to the extension fc'/fc: it is trivial if E/F is 
split and has order two otherwise. 

The local part of the main theorem of the paper is: 
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2.7.1. Theorem. Suppose chai(F) — char(fc) > max{n, 2} and r)E/F(Aa>b) = 1. 
Then there is an isomorphism of graded Frob k -modules: 

vaLv(A a ,i,) 

(2.7.1) H%MXt b ®kkMi{rik>/k)® l ) = H\N^® k kMz) 

i=0 

Taking Frobenius traces of the isomorphism (|2.7.ip , we get 

2.7.2. Corollary. Conjecture 12.5.21 hence Conjecture is true if char(F) = 
char(/c) > n. 

The proof of Theorem 12 . 7.11 will be completed in 33 We first prove an easy case. 

2.7.3. Lemma. Let SI — k ifk'/k is split, or CI — k' ifk'/k is nonsplit, we have an 
isomorphism of schemes over CI: 

val F (A Q , b ) 

(2.7.2) \\ M l °l b ® fc SI ^ ® k SI, 

8=0 

Proof. After base change to SI, we may assume that E/F is split. Then the ar- 
gument is the same as the proof of Lemma 12.5.51 once we fix an identification 
R a {E) = R a {F)®R a {F). □ 

2.7.4. Corollary. Theorem \K71\ holds if E/F is split. 

3. Global formulation — the moduli spaces 

Let k = F q be a finite field with char(/c) > max{n, 2}. Let X be a smooth, 
projective and geometrically connected curve over k of genus g. Let tt : X' — > X be 
an etale double cover such that X'/k is also geometrically connected. Let a denote 
the nontrivial involution of X' over X. We have a canonical decomposition: 

ir*O x > =O x ®C 

into ±l-eigenspaces of a. Here C is a line bundle on X such that C® 2 = Ox- 
Let D and Dq be effective divisors on X. Assume deg(D) > 2g. 

3.1. The moduli spaces associated to s n . Consider the functor M_ : Sch/k — > 
Grpd: 

£ is a vector bundle of rank n over X X S, 



s h-> < {£,<!>, a,m) 



Here, the twisting by (D) or (Dq) means tensoring with the pull-back of the line 
bundles Ox(D) or Ox (Do) to X x S. For each integer i, we define the subfunctor 
A4 i of by taking only those vector bundles £ such that 

\(X (g) k k(s),£ ® fe fc(s)) = i - n(g - 1) 

for any geometric point s of 5. It is clear that M. i is represented by an algebraic 
stack Mi over k locally of finite type; M is represented by M — U t Mi. 

Let Mf lt be the Hitchin moduli stack for GL„ (with the choice of the line bundle 
C(D) on X) which classifies only the pairs (£, 4>) as above. For more details about 
this Hitchin stack, we refer the readers to p~2l §4.2, §4.7]. 
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Let 

A := 0H°(X,£(D) 8 '); 

n-1 

B := @H°(X,O x (2D )®£(D)®*). 

i=0 

viewed as affine spaces over k. We have a natural morphism: 

fi : Mi -» A x B 

which, on the level of 5-points sends (£, (f>, ax, Px) to a = (a%, ■ ■ • , a n ) S A and 
b = (bo, ■ ■ ■ , b n -i) £ £> where 

a, = Tr(/\0) e x S, £(£>)»*) 

and h e x 5, Oxxs(2D ) ® £(C)® ! ) = Kom X xs{Oxxs(-D ), Ox*s(D ) ® 

C(D)® 1 ) is represented by the following homomorphism 

o XxS (-A)) ^ f ^ £ ® c(d)® 1 ^ OxxsW ® -CCD) 8 " 

3.2. The spectral curves. Following [101 §2.5], we define the universal spectral 
curve p : Y —> A x X as follows. For each S-point a = (ai, • • • , a n ) £ ^(S 1 ), define 
the following scheme, affine over X' x S": 

y Q ' := Spcc x , xS ^O x , x s(-iD)t^j 

where the ring structure on the RHS is defined by the relation 

t n - aif 1 - 1 + a 2 t n - 2 + {-l) n a n = 0. 

Let p' a : Y' a — > X' x S be the natural projection. This is a finite flat morphism 
of degree n. The scheme Y' a over X' x S naturally embeds into the total space 
Tot X 'xs(0(D)) of the line bundle O x >xs(D) over X 1 x 5. The free involution a 
on X' extends to a free involution on Y a by requiring a(t) = —t. The quotient of 
Y' a by a is the scheme 

Y * : =SE_xx S ^ -_pecxx5 (©A-^^SO^J . 

Let p a : y — > X x S 1 be the natural projection. This is a finite flat morphism of 
degree n. The scheme Y a naturally embeds into the total space Totxxs(CfD)) of 
the line bundle C(D) over X x S. The quotient map n a : Y' a — > Y" a is an etale double 
cover. 

Let „4 lnt (resp. A^ , resp. „4 sm ) be the open subset of A consisting of those 
geometric points a such that Y' a , and hence Y a , are integral (resp. reduced, resp. 
smooth and irreducible). Let B y = B — {0}. Let JvVf 1 be the restriction of Mi to 
A int xB x . 

3.2.1. Lemma. The codimension of A^ — A mt in A^ has codimension at least 
deg(D). 
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Proof. In our situation, A serves as the Hitchin base for GL„ and U„ at the same 
time (the unitary Hitchin stack will be recalled in §3.3|) . The locus .4 lnt is in fact 
the intersection of two elliptic loci: _4 mt = Aq L D A^] 1 . Here, Aq L is the locus 
where Y a is irreducible; A^] 1 is the locus where the set of irreducible components 
of Y' a is in bijection with that of Y a (cf. [TOl §2.8]. In the U„ case, the elliptic locus 
Aj\ is the 

same as the anisotropic locus considered in [12] . and by Proposition 
6.5.1 of loc.cit., we have 

codling (A* - A$ l J > deg(D). 

In the GL„ case, the same argument also works to prove that 

coding (.4^ - Aq L J > deg(D). 

In fact, we only need to compute the dimension of the Hitchin bases for the Levi 
subgroups GL ni x • ■ ■ x GL„ r . 
Therefore, 

dim(A^ - A mt ) = dim ((A^ - Afj U (A* - Af h J) < dimA^ - deg(D). 

□ 

The following lemma is a direct calculation. 

3.2.2. Lemma. For a geometric point a G A , the arithmetic genera of the curves 
Y' a and Y a are 

g Y :=l- X (Y^O Y ,) = n(n-l)deg(X>) + (2 5 -2)n + l; 
g Y :=l-x(Y a ,0 Ya ) = n(n - 1) deg(£»)/2 + (g - l)n + 1. 

Recall that for a locally projective flat family of geometrically integral curves C 
over S, we have the compactified Picard stack Vic(C/S) = JJ^ Vic (C/S) over S 
(see pQ) whose fiber over a geometric point seS classifies the groupoid of torsion- 
free coherent sheaves T of generic rank 1 over C s such that x(C s ,^) = i- Each 
"Pic (C/S) is an algebraic stack of finite type over S; it is in fact a <G m -gerb over the 
compactified Picard scheme of Pic* (C/S). The scheme Pic* (C/S) is proper over S 
and contains the usual Picard scheme Pic 1 (C/S) as an open substack. 

For each a £ .4 int (S) and T G Vic{Y a /S), the coherent sheaf £ — p a ,*J- is a 
vector bundle of rank n over X x S which is naturally equipped with a Higgs field 
<\> : £ -> £ ®Qx £( D ); conversely, every object (E,4>) G M mt (S) over a G .4 int (S) 
comes in this way. Therefore we have a natural isomorphism of stacks (cf. [5]) 

(3.2.1) Vlc~(Y/A iat ) ^ M mt \ Al ,t. 

Therefore we can view Ai\ nt as a stack over ViciY / A lnt ) ■ 

3.2.3. Lemma. The stack Mf^ represents the following functor 

a G A in \S),T eWc~ n(9 ~ 1] (Y a / S), 

(a, T ', a, [3) Ya (—Do) T ^> ui Ya /xxs(Do) such that 7 = (3 o a is 
nonzero along each geometric fiber of Y a — > S 

Proof. For a quadruple (a,!F,a,f3) as above, we associate (£ — p a ,*T,(f)) G Mf A 
by the isomorphism (|3.2.1[) . By adjunction, we have 

Uom x (Oxxs(-D ),£) = Romy a (O Ya (-DQ),F) 
Romx(£,Oxxs(D )) = Homy a (^ 

i ^Y a / X X S (Do))- 
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Therefore from (a, 0) we can associate a unique pair of homomorphisms 

O XxS (-D ) ^O XxS (D ). 

Let b = (bo, • ■ • , S B be the second collection of invariants of (£, 4>, A, /u). The 

composition 7 = /3 o a is an element in 

Eom Ya (O Ya (-D ),uj Ya/Xx s(D ))=Rom Xx s(O Ya ,O Xx s(2D )) 

which is given by 

n— 1 

(6 , • ■ • , 6n-i) : Oy. = C(-D)^ -> O XxS (2£> )- 

i=0 

Therefore for any geometric point s of £>, the condition b(s) 7^ is equivalent to 
that 7|y s ^ 0. □ 

3.2.4. Remark. From the proof of above lemma, we see that for any (a, b) g _4. lnt x 
£> x , the homomorphism 7 : Ya (— D ) —> LO Ya / X (Do) is independent of the choice 
of (T, a, (3) S -MJ 1 ^ b . We denote this 7 by 7^. Therefore we get a morphism 

coker( 7 ) : A int x B x -> Quot 2d (co(D )/Y/A int ) 
(a,b) h-> coker(7 ajb ). 

3.2.5. Remark. Let 

d = n deg(£\,) - n(g - 1) + g Y - 1 = n(n - 1) deg(L>)/2 + n deg(D ) 
By the moduli interpretation given in Lemma 13.2.31 vWf* is non-empty only if 
-d-n(g-l) = x(Y a ,O Ya (-D )) < i-n(g-l) < X {Y a ,u; Ya/x (D )) = d-n(g-l). 
(here a € A lnt is any geometric point); i.e., — d < i < d. 

3.2.6. Proposition. For —d + 2g Y — 1 < i < d— 2g Y + 1, Ai\ nt is a scheme smooth 
over k and the morphism f\ nt : M™ 1 — > A lnt x B x is proper. 

Proof. We have the following Cartesian diagram 

(3.2.2) M 1 ^ ^ Quot d - 1 {Lo{D )/Y/A mt ) 

AJ d _, 

Quot d+t (iv(D )/Y/A int )^^ ) h^~ ni9 ' 1) (Y/A int ) 
which, on the level of S'-points over a € A lat (S), are defined as 

(J 7 , a, (3) cokcr(JP" ^ uj Ya/XxS (D )) 

Q v (AJ d+i ) v T 

coker(uj Ya/XxS (D ) F y ) >■ T 

The only thing that we need to check is that coker(/3) and coker(a v ) are finite flat 
Og-modules of rank d — i and d + i respectively (when S is locally noetherian) . 
We check this for Q := coker(/3). For any geometric point s G S, the map 7 S : 

Ya s (—Do) T s — ^ oj Ya s / Xg (Do) is nonzero, hence generically an isomorphism. 
Therefore (3 S is surjective on the generic point of Y a _ s . Since T s is torsion- free of 
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rank 1, we conclude that (3 S is injective. Since u>Y a /xxs{Do) is flat over Os, we 
have 



Torf s (Q,fc(s)) = ker(/3 s 



0. 



This being true for any geometric point s € S, we conclude that Q is flat over Os- 
The rank of Q s over fc(s) for any geometric point ,s 6 S is 

xft.s^y.^/x.W)) - XtXt.s,-? 7 *.) =d-i. 

Since the Abel-Jacobi map AJ^-i is schematic and Quot d+l (uj(Dq) / Y / A mt ) is a 
scheme, .Mf 1 * is also a scheme. For i in the given range, we have d ± i > 2gy — 1, 
therefore (A3d+i) v and AJd-i are smooth by [H Theorem 8.4(v)]. Moreover, since 
we assumed deg(D) > 2g from the beginning, Vic(Y/A lrA ) = Af Hlt |_4int is smooth 
by [321 Proposition 4.12.1]. Therefore, A4J nt is a smooth scheme over k. 

We have a G m x G TO actions on M\ nt : (01,02) acts by changing (J 7 , a, (3) to 
(J 7 , c\u, C2/3). It is easy to see that (c, c _1 ) acts trivially so that the action factors 
through the multiplication map G m x G m — > <G m . This G m -action is free and the 
quotient M. i exists as a scheme. In fact, we can define M. i by a similar Cartesian 

-' n(g-l) lvl Aint , 



diagram as Q3.2.2p (the only difference is that the stack "Pic 
replaced by the scheme Pic^ 3 ~^ (Y/A [nt )): 



'{Y/A int ) is 



(3.2.3) 



M 



int 



}uot d - i (u)(D )/Y/A h 



Qnot d+l (cu{D )/Y/A 



int\ 



AJ rf 



^Pic (9 >{Y/A mt ) 



In other words, M. i classifies isomorphism classes of {T , a, f3) up to rescaling a 
and (3. It is clear that M™* is the quotient of VWf * under the above-mentioned 
G m -action so that A4J nt becomes a G m -torsor over A^™'. 

From diagram (|3.2.3[) . we see that M™ 1 is proper over A mt because the Quot- 
scheme Quot d+1 (u;(Do)/Y/A lnt ) is proper over A lnt and the morphism AJd-i is 
proper (with fibers isomorphic to projective spaces). Moreover we have a Cartesian 
diagram 



Afj nt — 

/! nt 

A int xB x 



Mi 

IF 



where the horizontal maps are G m torsors. Since both J^i i and A mt x P£> x are 
proper over A , the morphism /J nt is proper. Therefore f\ nt is also proper. This 
completes the proof. □ 

3.3. The moduli space associated to u„. Consider the functor N_ : Sch/fc — » 
Grpd: 

£' is a vector bundle of rank n over X' x S, 
h : £' ^ a*(£') v is a Hermitian form, i.e., a*h v = h, 
<j>' : 8' -> £'(£>) such that ct*<; 
// : £' -» Ox'xs(-Do) 



5* 



/! + /lO(A' = 0, 
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Here (— ) v — Hom Y (— , Ox)- It is clear that Af_ is represented by an algebraic stack 
N locally of finite type. 

Recall that we also have the usual Hitchin moduli stack A/" Hlt for U n classifying 
only the triples {£' , h, (/>') as above. For details about this Hitchin stack, we refer 
the readers to [10] . 

For (S',h,(j)',fi') eJ\f(S), since cr*0' v = -/iof oft- 1 , we have 

i 

a t = Tr(/\0') S H°(X' x ZOx.xsiiD))^- 1 ? = H°(X, £(£>)»*). 
Let A' = h^ 1 o (T*^i' v : Gx'xs(— A)) — > ct*£' v £'. Consider the homomorphism 

b'i : Ox'xs(-flo) £ ^ £'(iD) *U O x >y.s(D a + iD). 
We have a canonical isomorphism 

i:a*b'y = a*\' y o(a*(t) ,v y oa*h v o cr>' v 

= (-l)Vo^oA' = (-l)% 

such that cr*t v = l. Therefore b\ comes from a homomorphism 

h : O XxS (-D ) -» CW(A>) (8 /Ip) 01 . 

In other words, we may view &i as an element in H°(X x S 1 , Oxxs(2-Do) <8>£(^)® 4 )- 
The map that sends (£' , h, (/>' , \') G A/"(5) to a = (ai,-- - , a„) G .4(5) and 6 = 
(6o, • • • , & n -i) G defines a morphism 

3 : A/" ^ .4 x B. 

Let W int be the restriction of N to .4 int x B* . Similar to Lemma l3~2~3l we can 
rewrite A/" mt in terms of spectral curves. 

3.3.1. Lemma. The stack A/" lnt represents the following functor 

a G ^ int (5),^' G Wc{YjS), h : P ^ a*{F v ), 



(a,P,0>) 



such that cr*/i v = h, and JT' — > ojy' /xxs(Dq) 

which is nonzero along each geometric fiber of Y' a — » S* 



For (.T, /i, /?') G M^(S), let 7 ' = /?' o a' where a 1 = h' 1 o a*p' v : O Y >(-D ) -» 
J 7 '. Then 7' G Homy a /(0Y a /(-D o ),wy aVXxS (L>o)) satisfies cr*7 /v = 7'. Therefore 7' 
comes from 7 G Homy a (0y o (-Do),wy o /xxs(fo)) via pull-back along the double 
cover TT a : Y' a —> Y a . It is easy to see that this 7 is the same as the j a ,b defined in 
Remark 

3.3.2. Proposition. Suppose n(deg(Do) — g + 1) > <?y. TTien i/ie stac/c 7V mt is a 
scheme smooth over k and the morphism g lnt : Af lnt A lnt x B v is proper. 

Proof. By the moduli interpretation given in Lemma 13.3.11 we have a Cartesian 
diagram 

(3.3.1) Af int 3- Quot 2 V(A))/y'A4 int ) 

AJ 2 d 

^Hit Uint u—^-X)^,^^ 
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which, on the level of S'-points over a G A int (S), are defined as 
{P, h,/3') 2 C0 ker(^' lo y , /XxS (D )) 

r AJ 2d 

(P,h) u - >p 

Here T' has Euler characteristic —2n(g — 1) along every geometric fiber of Y' a — > S 
because T' = a*T N . 

bmcc u is schematic and Quot 2d {uj(D )/Y' /A int ) is a scheme, we see that A/" mt is 
a scheme. By [TUJ Proposition 2.5.2, Proposition 2.8.4], A/" Hlt Uint is smooth over k 
and proper over .4 lnt . Moreover, since d = ndeg(Do— g+l)+gy — 1 > 2gy — 1 = <?^, 
2rf > 2g' Y , therefore AJ2d is smooth by [I] Theorem 8.4(v)]. Therefore Af mt is a 
smooth scheme over k. 

We have a G m -action on 7V mt by rescaling j3' . Unlike the case of A^ nt , this 
action is not free: the subgroup ^2 C G m acts trivially. The quotient of J\f mt 
by this G m -action is a Deligne-Mumford stack AT proper over A/" Hlt |^mt, hence 
proper over A mt . Let (P£> x )' be the quotient of B y by the square action of the 

dilation by G m . This is a separated Deligne-Mumford stack. Therefore AT" 1 * is 
proper over .4 lnt x (FB X )'. We have a Cartesian diagram 

Af int ^AT int 

gint ^ 

A int x B x ^int x (pgxy 

This implies that g int : J\f int — ► ^l int x B x is also proper. □ 

3.4. The product formulae. We want to express the fibers of /? nt in terms of 
local moduli spaces similar to AA 1 "^ b defined in W2.3{ To simplify the notations, 
we only consider fibers M b where (a,b) € A mt (k) x B x (k), but the argument is 
valid for (a, b) G A int (fl) x B x (fi) for any field Q D k. 

Let \X\ be the set of closed points of X. For any x G \X\, let Ox, x be the 
completed local ring of J at 1 with fraction field F x and residue field fc(x). Then 
Ox,x is naturally a fc(x)-algebra. Let Ox',x be the completed semilocal ring of X' 
along tt^ 1 (x) (recall Notations II. 6. 3ft and let E x be its ring of fractions. Then E x 
is an unramified (split or nonsplit) quadratic extension of F x . 

Fix an cr-equivariant trivialization of Ox'(D) along SpecOx'.xj which allows us 
to identify £|spcce>x x with . Fix a trivialization of Ox{Dq) over SpecOx^- 
Using these trivializations, we can identify the restriction of (a, b) on Spec Ox, a 
with a collection of invariants a x = (ai jX) ■ ■ ■ ,a 7l!X ) and b x = (bo. 
such that ai iX ,bi lX G (Cb x ) <t= '~ 1 ^- Using F x , E x and (a x , 6 X ) in place of F, E and 
(a, 6) in the discussion of ij2.2l and l2.3i we can define the 0f x algebra R ax , which is 
isomorphic to Oy a . x (see Notation ll.6.3p . The trivializations also identify R y a ^ with 
u;y a/ x(A))|spoce>y a , x and 7 ax! b x : fl „, -^ i^ (defined in (|2.2.4|) with 7 a ,6|s P ccCV, x 
(defined in Remark I3.2.4|) . As in (|2.4. 1 jl . R an .(E x ) has a natural Hermitian form 
under which i? ax (0E x ) and i?^ (Cb x ) are dual to each other. 
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With these data, we can define the local moduli spaces of lattice A4'° c ax bx (i x E 
Z) and A/2° C f, as in 32.31 and 12. 4[ which are projective schemes over k(x). To 
emphasize its dependence on the point x, we denote them by a b and J\f* 



■Ij.r ' 



3.4.1. Proposition. For (a,b) € .4 lnt (fc) x B x (k), there is an isomorphism of 
schemes over k: 

(3.4.1) Mi, a , b ^ ]J II Res Hx)/kM x ln: ^ : 

Here \X\ is the set of closed points of X , Res/.( x \/fc means restriction of scalar s. 
The disjoint union is taken over the set of all assignments x £ \X\ i — > i s 6 Z such 
that 2xe|x| ■ k]i x = d — i. The product is the fiber product of schemes over k. 

Proof. First we remark that the RHS in the isomorphism (|3.4.1[) is in fact finite. 
Note that for each x 6 \X\, there are only finitely many i x such that A4f a b 
is non-empty (more precisely, < i < val^A^,^)). For fixed (a, b) G .4 mt (&:) x 
B x (k), the map ^ a ,b ■ ^V a ( — -Do) — » uj Ya /x{Do) is an isomorphism away from a 
finite subset Z C and when ^ x is an isomorphism, Aif a ^ &x = Specfc(x). 
Therefore we can rewrite the RHS of (|3.4.ip as a finite disjoint union of 

II ' ReB H*)/k M t,a.,b m > 
xez 

which makes sense. 

By the Cartesian diagram (|3.2.2p . the assignment (J 7 , a,f3) h- > coker(/3) defines 
an isomorphism of schemes 

Mi, a ,t = Quot d -\Q/Y a /k). 

where 2 = coker(7 a Since j a ,b is an isomorphism on X — Z, Q is supported on 
Z x S. Therefore we get a canonical decomposition 



2 = 02, 



with each Q x supported at x. We get a corresponding decomposition of the Quot- 
scheme 

Quot d - l (Q/Y a /k) £ [] [J] Res kix)/k Quo& (Q x /R a Jk{x)) j . 



To prove the isomorphism (|3.4.1j) . it remains to identify Quot ltc (Q x /R ax /k(x)) with 
jM? Qx bx , but this is obvious from definition. □ 

Similarly, we have 

3.4.2. Proposition. For (a.b) E A mt (k) x B x (k), we have the following isomor- 
phism of schemes over k: 

(3-4.2) N a , b = [] Res fe(x)/fe A/^. 

xe|x| 
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3.4.3. Corollary. For any geometric point (a,b) G A lnt (Q) x B x (fl), there is an 
isomorphism of schemes over Q: 

d 

i= — d 

Proof. This follows from the two product formulae and Lemma 12.7.31 □ 

3.5. Smallness. Recall that in [12] Ngo defines the local and global Serre invariants 
for points on the Hitchin base. In our case, for a geometric point a 6 A(fl), let 
Y a — > Y a be the normalization. Then the local and global Serre invariants are 

5(a,x) = ^vaa{Oy a JO Ya ,x); 

5(a) = dimn H°(Y a , Y JO Ya ) = ^ S ( a > x )' 

xex(n) 

3.5.1. Corollary. For any geometric point (a, b) 6 („4 mt x B x )(£l), we have 

(3.5.1) dimo7Va : 6 = sup dimn M^ a ,b < 5(a). 

i 

Proof. The first equality follows from Lemma l2.7.3l Now we prove the inequality. 

For each x 6 X(fl), M. x ia b is a subscheme of the affine Springer fiber of 
GL„ associated to a regular semisimplc clement with characteristic polynomial 
a x . On the other hand, 5(a, x) is the dimension of that affine Springer fiber (cf. 
Bezrukavnikov's dimension formula [3), and [T^l §3.6, 3.7]). Therefore, 

(3.5.2) dim n Ml a ^ bx <S(a,x). 

Now the inequality (|3.5.1[) follows from the product formula (13.4. ip (note that we 
only formulated the product formula for fc-points (a, b) of A lnt x B x , but it has an 
obvious version for any geometric point of A lnt x8 x .) □ 

For each S > 0, let A- s (resp. A- 3 ) be the open (resp. closed) subset of A lnt 
consisting of those geometric points a such that 6(a) < 5 (resp. 6(a) > 6). Recall 
the following estimate of Ngo ( [T2l Proposition 5.4.2], based on local results of 
Goresky-Kottwitz-MacPherson on the root valuation strata [6]). For each 6 > 0, 
there is a number cs > 0, such that whenever deg(D) > cs, we have 

(3.5.3) codim^gt(^ e ) > e, VI < e < 6. 

Here we write A 1 ^ to emphasize the dependence of ^4 mt on D. 
Finally, we prove the smallness. 

3.5.2. Proposition. Fix 6 > 0. For deg(D) > c s , n(deg(D ) - g + 1) > 6 + g Y , the 

morphisms 

f^ 5 : Mf* = MT'U^s -^A^ s xB y V-d<i<d 
g <s . jj<s =AA int |_ A < 5 -» A^ 5 x B x 

are small. 

Proof. First, by Corollary 13.4.31 for any geometric point (a, b) £ A lnt x B x and 
any integer i, dim Adi ia ,b £ dim Af a ,b, therefore the smallness of g— implies the 
smallness of ff S . 
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Now we prove that g- s is small. For each j > 1, let (A lnt x B x )j be the locus 
where the fiber of g\ nt has dimension j. By Corollary 13. 5. 11 we have 

(„4 in * x6 x ) J ci- , x6 x . 

In particular, (A- s X B x )j is nonempty only if j < 6, 

Claim. For every geometric point a € A lnt (£l), the morphism g™ 1 : A/"" 1 * — > S x Cg)fcf2 
is dominant and generically finite when restricted to every irreducible component 

Proof. To simplify the notation, we base change everything to Spec (f2) via a £ 
„4 m *(f2) and omit from the notations. 

Let B a C B be the open subset consisting of geometric points b such that 7 a ,fc : 
Oy a {— Do) — > wy o /x(Do) is an isomorphism at all singular points of Y a . It is clear 
from the product formula (|3.4.2p that if b G B a , then N a ,b is finite and nonempty 
over f2. 

Note that by the Cartesian diagram (|3.3.ip . the morphism r : TV" 1 ' — > TV™* is 
smooth with fibers isomorphic to punctured vector spaces, therefore each irreducible 
component C of A/"™* is of the form r -1 (C") for some irreducible component C 
of TV™*. By [HI Corollaire 4.14.3], TV™* is equidimensional, therefore so is TV^ nt . 
Therefore, it suffices to show that for each irreducible component C of TV" 1 * , we have 
g™ t (C)nB a ^ 0. Because then, <?jf*|c must be generically finite onto its image; but 
by the equidimensionality statement, we have dim C = dimA/"" 1 ' = dim B a = dim B, 
therefore C'(C) = B x . 

Now we fix an irreducible component C C A/^ n * of the form r _1 (C"), where 
C" C TV™* is an irreducible component. We argue that g^iC) R B a ^ 0. By f!2l 
Proposition 4.14.1], the locus of (J-',h) £ C" where J 7 ' is a line bundle on Y' a is 
dense. Let (JF', h) e C" be such a point. Since 3-^ is embedded in a smooth surface, 
hence Gorenstein, u>y'/x is a line bundle on Y' a , hence T N is also a line bundle. 

Let Z be the singular locus of Y a . By the definition of 5(a), we have 

#z < 2dimH°(y a ,e> ?a /(rvj = s{a). 

where Y a is the normalization of Y a . 

To show that 5i n *(C)n£ a ^ 0, we only have to find (3' G Hom(J"', w r< / /x (Z)o)) = 
H°{Y^T' V (D )) such that coker(/3') avoids the singular locus Z' = ^(Z) of 
Y' a , since the support of coker(7) is the same as the projection of the support of 
coker(/3'). 

Consider the evaluation map 

O^K^^(D )^X 0QO/) -o. 

y'EZ' 

Note that 

X(XM = xK,^' v (7J )) - #Z' > 2n(deg(7J ) - <? + 1) - 25(a) > 2g Y > g' Y . 

Therefore, by Grothendieck-Serre duality, H 1 (Y^,K.) = Homy/(K,wy) v = 0. Hence 
the evaluation map 

H\Y>,^{D )) n(y>) 

y'ez' 
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is surjective. In particular, we can find (3' £ H°(Y^, JF' V (Z? )) which does not vanish 
at points in Z 1 . This proves the claim. □ 

Applying the above Claim to the geometric generic points of A (note that 7V mt — » 
A mt is surjective because A/" Hlt l_4i.1t — > „4 mt is), we see that g lnt restricted to every 
geometric irreducible component of A/" lnt is generically finite and surjective. 

Using the above Claim again, we see that for any geometric point a £ A mt and 
j > 1, the locus of b £ B x where dim Af a ,b = j has codimension > j + 1 in B x . 
Therefore, 

codim_ 4 > JxS x (7l int x B x )j >j + l. 
Since deg(D) > c$ and j <S, we have 

codim_4i„t xf 3x (^l mt x B x )j 
> codim_4int(^4- : ') + codim_4>i xB x („4 int xB x )j >2j + l. 
This proves the smallness. □ 

4. Global formulation — matching of perverse sheaves 
4.1. A local system on Ai\ nt . Consider the morphism 

(4 ' L1) 

v : MT ^ Cinot d - l (uj{D )/Y/A mt ) > Syrn^ (Y/A int ) -> Sym^pT/fc) 

where V\.Y/A lat 1S the norm map defined by Grothendieck in [8]. Here Syni d ~* (Y/ A lnt ) 
(resp. Sym d ~ l (X / k)) is the (d — i) th symmetric power of Y over ,A mt (resp. X 
over k), constructed as the GIT quotient of the fibered power (Y/A ult ) d ~ l (resp. 
(X/k) d ~ l ) by the obvious action of the symmetric group T^d-i- The morphism 
Sym d ~ l (Y/A lnt ) — > Sym d ^ l (X/k) above is induced from the natural projection 
(Y/A int ) d - 1 -» {X/k) d -\ 

We will construct an etale double cover of Sym d_i (X/fc), which will give us a 
local system of rank one and order two on Sym d_l (X/fc), hence on M.™* 1 . 

The groups (Z/2) d ~ 4 and £d_i both act on (X' /k) d ~ l and they together give 
an action of the semidirect product T = (Z/2) d ~ l x E<j-i on [X' /k) d ~ % . Then 
Sym d ^\X/k) is the GIT quotient of (X'/k) d ^ i by T. We have a canonical surjective 
homomorphism e : T — > Z/2 sending («i, • • ■ ,i>d_i, s) 1— > «! + ••• + Ud— where 
Wj e Z/2 and s e £d-i- Let To be the kernel of e. 

4.1.1. Lemma. Let Z d ~ l be the GIT quotient of{X'/k) d - 1 by T . Then the natural 
morphism £ : Z d ~ l — > Sym d_l (X/fc) is an eta/e double cover. 

Proof. Let i = + • • • + Xd-i be a geometric point of Sym d_I (X) and let t' — 
{x[,--- ,x' d _j) be a geometric point of (X' /k) d ~ l over t. Then the point t" — 
(a(x' 1 ),x' 2 , ■ ■ ■ >Xd-i) 1S an °ther geometric point of (X' /k) d ~ l which does not lie in 
the r -orbit of t' . Therefore t' and t" have different images z' and z" in Z d ~ % . In 
other words, the reduced structure of C" 1 W consists of (at least) two points z', z". 
Consider the maps 

£ : {X'/k) d - 1 ^ Z d - 1 ^ Sym d -\X/k) 

where £ is finite flat of degree 2 d ~ l (d — The degree of the geometric fibers 
n~ 1 (z') and r/^ 1 (z") are at least 2 d ~ l ~ 1 {d — i)\ (because this is the generic degree). 
This forces the two degrees to be equal to 2 d ~ I ~ 1 (d — This being true for any 
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geometric point of Z d l , we conclude that the quotient map (X'/k) d 1 — > Z d 1 is 
flat, hence faithfully flat. Therefore C : Z d ~ l -> Sym d ~ l (Jf/fc) is also flat (of degree 
2). Since every fiber £ _1 (a;) already consists of two distinct points, these two points 
must be reduced. Therefore £ is finite flat of degree 2 and unramified, hence an 
etale double cover. □ 

Let L^_ i be the local system of rank one on Sym d ~ l (X / k) associated to the etale 
double cover Z d ~ l -> Sym d ~* (X/fc) ( cf - Notation 03) . We define 

Ld-i ■= v*L^_ i 

to be the pull-back local system on M\ nt . 

We describe the stalks of the local system L^_ i in more concrete terms. Let L be 
the local system of rank one on X associated to the etale double cover ir : X' — > X. 
Let Co = ker((Z/2) d_ * ^> Z/2). Then (X'/k) d -' l /C is an etale double cover of 
(X/k) d -\ 

4.1.2. Lemma. 

(1) We have a Cartesian diagram 

(x'/ky-yco » z d - 1 



(X/k) d - 1 ^ Sym d - l (X/k) 

where the maps are all natural quotient maps. 

Proof. (1) follows from the fact the both vertical maps are etale (Lemma 14.1. ip . (2) 
The local system of rank one associated to the etale double cover (X' / k) d ~ % / Cq — » 
(X/k) d -' 1 is clearly L m{ - d ~^ . Therefore (2) follows from (1). □ 

4.2. The incidence correspondence. Recall that _A sm is the open locus of a 6 
A mt where Y a is smooth (equivalently, the locus where 8(a) — 0). We assume that 
deg(D) is large enough (> ci) so that ^l sm is nonempty, hence dense in A lnt . The 
norm maps Olyz/^int and OTyy^mt are isomorphisms over ^l sm . 
For each —d < i < d, consider the incidence correspondence 

T d-i,2d c s ym d - 4 (r/^ sra ) x Sym 2d (y/^ sm ) 

whose geometric fiber over a E A sm classifies pairs of divisors T < T' on Y a 
where deg(T) = d— i, deg(T') = 2d. Let t, t' be the projections of l d ~ l ' 2d to 
Sym d - 1 {Y/A sm ) and Sym 2d (y/^ sm ). 

4.2.1. Lemma. The incidence correspondence J d ~ l ^ 2d { s smooth over A sm . 

Proof. Since Y —> A srVL is a smooth family of curves, we may identify the symmetric 
power Sym' (Y/A sm ) with the Hilbert scheme Hiib 7 (Y '/ A sm ) . We apply the infin- 
itesimal lifting criterion to prove the smoothness. Suppose Ro is a local artinian 
ring and R is a thickening of Rq. Let a 6 _4 sm (i?) with image ao 6 .4 sm (i?o)- Let 
T C Tq C Ya be subschemes of flat of degree d — i and 2c? over i? - We want to 
find subschemes T C T' C Y a , flat of degree d — i and 2c? over R, whose reduction 
to Rq are precisely To C Tq. We may assume that Tq is contained in an affine open 
subset U C Y a . Let U = U n T ao . Since Y^ is sl smooth family of curves over 
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Ro, the subschemes To and Tq arc dcfiricd by the vanishing of functions fo and 
fo G r(J7o, Oc/q) respectively. Since To C Tq, we have f' Q = fogo for some function 
go G r(Z7o, Ou )- Let /, 5 be arbitrary liftings of /o,<7o to r(£7, Ojj) (which exist 
because U is affine), define T and T' to be the zero loci of / and fg. Then it is 
easy to check that T C T' C Y a are flat over R of the correct degree. This proves 
the smoothness of I d - l > 2d over „4 sm . □ 



We define a morphism 

lint k>X cokcr 7 



div : A mt x B" 



iMf..(n Aint\ ^y/a^ t , 2,/ 



Quot' d {to (D ) /Y/A u 



Sym M (Y/A mt ) 



which sends (a, b) to the cycle of coker(7 Q) b) in Y a 
4.2.2. Lemma. We have a Cartesian diagram 

div 



(4.2.1) 



/r 

.4 sm x B y 



d-i,2d 



div 



Sym 2d (Y7.4 sm ) 



Here the morphism div : Aif 11 — > 7 d l,2d over a point a G ,4 sm sends (J-, a, (3) to 
the pair of divisors div(/3) C div(a, 6) 0/ K a . 

Proof. We abbreviate Quot 2d (cj(T> )/T/„4 int ) by Quot 2d . Let Q be the universal 
quotient sheaf on Y x^mt Quot 2d . Then by the moduli interpretation given in 
Lemma 13.2.31 we have a Cartesian diagram 



/i nt 

^ int x B y 



r 



Quot* - * (Q/(T x_4i„ t Quot 2d )/Quot 2ci 



cokcr(7) 



Quot 



2./ 



Restricting this diagram to „4 sm x B x , we have 

Quot 2< V"> = Sym 2d (y/^ sm ) 



)uot 



d - 1 (Q/(Y x^» t Quot 2d )/Quot 2d ) I 



1 M ( 
rd—i,2d 



because the norm maps Wly/A'™ are isomorphisms. Therefore the diagram (|4.2.ip 
is Cartesian. □ 



Let L Y d _ i be the pull-back of the local system L d _ i via Sym d l (Y/A 



Syiri (X/fc). Define 



t'^LU G T; c b (Sym 2rf (TM sm ),Q,). 



4.2.3. Lemma (Binomial expansion). Let n% d : Sym 2<i (Y7„4 sra ) -> Sym 2d (Y7„4 sm ) 
6e i/ie natural projection. Then there is a natural isomorphism 

d 



(4.2.2) 
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Proof. Since the morphism r' : 7' 



!-i,2d 



Sym 2d (y/^ sm ) is finite and T 



d-i.2d 



IS 



smooth over k by Lemma [4.2. H we conclude that Kj_ i} being the direct image of a 
local system under r', is a middle extension on Sym 2d (F/.4 sm ). Similarly, 7r^ is 
also a middle extension on Sym 2d (y/^4 sm ). Therefore, to establish the isomorphism 
(|4.2.2jl , it suffices to establish such a natural isomorphism over a dense open subset 
of Sym 2d (Y/A sm ). 

Now we consider the dense open subset U C Sym 2d (y/^l sln ) consisting of those 
divisors which are multiplicity- free. Let U' (resp. U, resp. U') be the preimage of 
U in Sym 2d (yy.4 sm ) (resp. (Y/A sm ) 2d , resp. (Y'/A sm ) 2d ). Then s 2d : U -► U is 
an etale Galois cover with Galois group T,2d, and we have a Cartesian diagram 



E/' ^C/' 



s 2d , 

[/ »-l7 



Therefore we have 

where L y is the pull-back of L to Y\ 

On the other hand, we have a Cartesian diagram 



UjC{l,2,- ,2d},#J=d-i Uj *■ I d l ' 2d \u 



u- 



u 



where for each J C {1, 2, • ■ • , 2d}, Uj C Sym d_4 (Y/„4 sm ) x U is the graph of the 
morphism tj : (yi, • • • , y2d) >— > Uj- Therefore we have 



i s 2d^d-i)\u — ( s 2d T '* T * Ld-i)\u — 

JC{1,2,--- ,2d},#J=d-i 



T*T Y 



For each J C {1, 2, • • • , 2d} of cardinality d - i, let pj : U -> (Y/.A sm ) J be the 
projection to those coordinates indexed by J. Then we have a factorization 

Tj:U ^ {Y/A am ) J ^ Sym d - i (Y/A sm ). 
By Lemma mOJ we have s}L^ L^^"*). Therefore 

T J L d-i — V,l s J L d-i — Pj\\ h ) )■ 

Finally, we have an E2d-equivariant isomorphism of local systems on U: 
(4dKU\u = Pj((L Y f* J ) = (Qe © L Y T*%. 

i=-d JC{1,2,- - ,2d} 
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The last isomorphism justifies the nickname "binomial expansion" of this lemma. 
This E2rf-equivariant isomorphism descends to an isomorphism 

d 

i— — d 

which proves the lemma. □ 

4.3. A decomposition of ,g' nt Qf. To state the next result, we need to define a 
technical notion. 

4.3.1. Definition. A commutative diagram of schemes 
(4.3.1) 




is said to be pointwise Cartesian, if for any algebraically closed field Q, the corre- 
sponding diagram of Jl-points is Cartesian. 

For pointwise Cartesian diagrams with reasonable finiteness conditions, proper 
base change theorem also holds: 

4.3.2. Lemma. Suppose we have a pointwise Cartesian diagram U^SAd where all 
maps are of finite type and f,f are proper. Let F £ D h c (X, Q^) be a constructible 
^-complex on X , then we have a quasi-isomorphism 

P*f,F = fla*F. 

Proof. Let X" — Y'x Y X and /" : X" -» Y' be the projection. Let £ : X' -> X" 
be the natural map over Y' , which is also proper. By the usual proper base change 
for Cartesian diagrams, we reduce to showing that for any G £ D b c {X" , Q e ), 

(4.3.2) fl'G S f(CG. 
But we have 

fU*G = /,"£*rG = f'J(G ® m e ). 
Therefore to show (|4.3.2p . it suffices to show that the natural map l : Q e — ► £*Q^ 
is a quasi-isomorphism. Since both £*Q^ is constructible, it suffices to show that i 
is an isomorphism on the stalks of every geometry point x" £ X" i.e., 

(4.3.3) l x „ :® i ^H*(C 1 (x"),®t) 

is an isomorphism. By Definition 14.3. 1[ (x")(Q) is a singleton. Therefore, the 
reduced structure of £,^ 1 {x") is Specfi, and (|4.3.3[) obviously holds. □ 

Consider the norm map 

9V, M i„t : Quot 2d (Lu(D )/Y'/A int ) -» Sym 2d (y7^ int ). 

4.3.3. Lemma. The diagram 

jysm L Sym 2d (r7^ sm ) 



A sm xB x di^_^ S ym 2d (Y/A Snl ) 
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is pointwise Cartesian. 

Proof. Since A/" sm is reduced, to check the commutativity of the diagram, it suffices 
to check on geometric points. Therefore, we fix a geometric point (a, b) € A sm (il) X 
B x (fl), and prove that the diagram is commutative and pointwise Cartesian at the 
same time. Again we omit fl in the sequel. 

Let div(a, b) = Xw— 1 m tUt with {yt} distinct points on Y a . Recall 7r a : Y' a — > Y a 
is the etale double cover induced from it : X' — > X. Let TTa 1 {yt) = {y'tiUt}- Then 
a point Oyi(-Do) C f C ^y//x(.Do) is determined by the torsion sheaf Q' = 
^Y' /x(Dq)/ J 7 ' , which is a quotient of uiyr /x(Do)/Oy'(—Dq), Since Y£ is smooth, 
Q' is in turn determined by its divisor X)t=i rn tyt~^ rn ty't — ^a 1 (div(a, 6)). The line 
bundle T 1 is self-dual if and only if m' t + m" = m t (cf . the proof of Lemma I2.5.6| . 
Therefore, the image of the divisor of Q is div(a, b). This proves the commutativity 
of the diagram. But this also shows that the map M a ,b 7r M _1 (^ v ( a > ^)) ^ s a 
bijection on il-points. This completes the proof. □ 

4.3.4. Lemma. Fix d > 1, deg(D) > c$ and n(deg(D ) — g + 1) > 5 + gy . Then 
(f ) For each —d < i < d, div* Kj^dimA + dim£>] is a perverse sheaf. 
(2) We define 

Kd-i ■= iff (div*#J_ i [dim.A + dimB])[- dimA - dimB] 

where j sm : _4 sm x B x *—* A- 13 x B x is the open inclusion. Then we have a 
natural decomposition 

d 

gf 5 Q? = K d _i. 

%— — d 

Proof. (1) By Lemma T4 . 3 . 3 1 and Lemma f4.3.2l we have 

Since g sm is finite and A/" sm is smooth, gl m Qf[dim A + dimS] is a perverse sheaf. 
But by Lemma [4.2.31 Kj_ { is a direct summand of 7r|^ ^Q e , hence div*.KT ; is a 
direct summand of div*7r|^ t Q e = gl m Qe- Therefore div* Kj^dim A + dimS] is 
also a perverse sheaf. 

(2) follows from the smallness of g- s proved in Proposition 13.5.21 □ 

4.4. The global matching theorem. The global part of the main theorem of the 
paper is: 

4.4.1. Theorem. Fix 5 > 1 ,deg(D) > c s and n(deg(L> ) - g + 1) > 5 + gy- Then 
for —d < i < d, there is a natural isomorphism in D C (A— x B x ): 

(4.4.1) f^Ld-i = K d -i, 

hence an isomorphism 

d 

f^L d ^=gf% t . 
i=—d 

The proof of the theorem will occupy the rest of the section. We first prove a 
relatively easy case of the theorem. 

4.4.2. Lemma. Theorem\j4J\ holds if -d + 2g Y - 1 < i < d - 2g Y + 1. 
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Proof. For 5 > l,deg(D) > c$, n(deg(Do) — g + 1) > 6 + gy and i in the above 
range, by Proposition 13.2.61 and [373721 A4\ nt and 7V mt are smooth. Moreover, by 
Proposition ^ . 5 . 2l the morphisms ff are small. Therefore both Ld-i and K d -% 
are middle extensions on A- 5 x B x . Hence it suffices to establish the isomorphism 
(14.4. fj) on the dense open subset .4 sm x B x C A- 5 x B x . 
The morphism v (see (|4.1.ip ) restricted on «Mf m factors as: 

i/ sm : M! m ^ I d - tad A Sym^^y/^ 3111 ) -» Sym^^X/fc). 
Therefore, we have 

T I ,, sm >* T A rv j- T * * tY 

L d -i\Mf = v L d _i = div r 
Applying proper base change to the Cartesian diagram (|4.2.f \i , we get 

./r: : /-/ - = f%tiv>Ld-i = divXr*^ 

□ 

Now we deal with the general case. Choose a closed point oo e \X\, let A°° be 
the open subset of A mt where p a : Y a — » X is etale over oo; let (.A mt xB x )°° be the 
open subset of A°° xB x consisting of points (a, 6) such that -f a ^ is an isomorphism 
over 00 (or equivalently, div(a, b) avoids 00). If we vary 00 G \X\ (even if we are 
restricted to those 00 which are split in X'), the open sets (A mt x B x )°° obviously 
cover .A lnt x B x . Now the theorem reduces to the following Proposition. In fact, by 
this Proposition, for all — d < i < d, f^Ld^i\^<s x ^x^^ is then a middle extension 

on (A- s x B x )°°. Since this is true for any split 00, we conclude that ff^Ld-i is a 
middle extension on ^4— ^ x B x . Then we can apply the argument of Lemma T4. 4. 21 
to finish the proof of Theorem 14.4.11 

4.4.3. Proposition. Suppose 00 is split in X' . Then for each — d < i < d, the 
complex ff^ Ld-i\(A<s x b x ) <x ' * s a middle extension on (A- s x B x )°° . 

Proof. For each N > 0, let Dn — Dq + Noo be a new divisor on X. We can use Dn 
instead of Dq to define the various moduli spaces and they enjoy all the properties 
we have proved so far. For those moduli spaces that depends on the choice of 
Dn, we add subscripts "Do" or "Dat" to emphasize the dependence. Consider the 
embedding 

l n ■ (A int x B x o )°° ^ int x B x o ^ A hlt x B x n 

given simply by the embeddings H°(X, O x (2D a ) ®C(D)^) <^> H°(X, O x {2D N )® 
C{D)® 1 ) induced by the natural embedding O x (2D ) ^> O x (2D N ). Recall from 
Remark 13.2.51 that we use D and Dq to define an integer d = n[n — 1) deg(D)/2 + 
ndeg(A))- Let d N :=n(n- 1) deg(D)/2 + ndeg(D N ) = d + Nn[k(oo) : k}. 

Let p _1 (oo) is the preimage of 00 in the universal spectral curve Y°° — > A°°, 
and 2Np~ 1 (oo) is viewed as a flat family of subschemes of Y°° —> A°° , the 2N th 
infinitesimal neighborhood of p _1 (oo) in Y. For each < ioo < 2Nn[k(oo) : k], 
consider the Hilbert scheme of the scheme 2Np~ 1 (oo) over A°°: 

h loo : Eilb^(2Np- 1 (oo))/A oc ) A°°. 
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We claim that for any — < j < d,N we have a Cartesian diagram 
IJ M% oA x A ~ mb l -(2Np- 1 (oo))/A^) 



(4.4.2) 



(^ int x B5J 



where the disjoint union is over the set i + (A^n — i oa )[k(oo) : k] = j and < iao < 
2Nn; the morphism fg> o4 : M<fi 0>i -> (A int x B£ )°° is the restriction of f% o i . 
In fact, the verification of this Cartesian diagram is same as proving the product 
formula [3.4.1[ except here we are working over the base (^4 mt x Bp )°° rather than 
over a point. 

We also have a commutative diagram relating the maps v (see the beginning of 
§4.11) in the situation of Dq and Dn- 



M D , t x -4~ Hilb io °(2Afp- 1 ( 00 ))M °) 



-lf^W/ Aoo\ 'iv(Moo) . , int 

*~ JVl D N ,j 



Sym d - t {{X -oo)/k) 



+icxD oo 



Sym dN - j (X/k) 



Since oo is split in X', the pull-back of L to Specfc(a;) (via x) is a trivial local 
system. Hence by Lemma \A. 1.21 we have (+i 00 oo)*L^ N ■ = L^^ Therefore 

TN(i,ioo)*L dN -j =T N (i,i 00 )*Vn N L* N _ j = v|, o (+i 00 oo)*i^_ j = L d ^ M A °c Q e . 
Applying proper base change to the diagram 14. 4. 21 we have 



(4.4.3) 



* fiat j 
L NJD N ,j.* Ij d. N -j 



e 



Tl 



i+(Nn-i ao )[k(<x>):k]=j 

Since oo is split in X' , we fix a point oo' 6 \X'\ over oo and let p /_1 (oo') be its 
preimage in the family of curves Y'°° — > .4°° . Similarly we have a Hilbert scheme 
for < ioo < 2Nn[k(oo) : k]: 

ti lca : m\b i °°{2Np'- 1 {oo'))/A ao ) -> A°°. 
We also have a Cartesian diagram 



(4.4.4) 



- \f int 

yV -Div 



9o, 



A int x Bl 



Uil=o^ x ^ Hilb'~(2JVp'- 1 (oo'))M°°) - 

{A mt xBy°° 

where : A/qJ — > (A mt x £>^, o )°° is the restriction of g 1 ^. This diagram can be 
verified similarly using the argument of the product formula [3.4.2[ and notice that 
since oo is split in X', we have the isomorphism (|2.7.2p by Lemma [2. 7.31 which also 
works over (,A int x B x )°° to justify the appearance of Hilb l °° (2A^p' _1 (oo'))/^°°) in 
the above diagram. Applying proper base change to the diagram 14.4.41 we get 

2Nn 

(4.4.5) iN^Jk = 9d ,M EU- h' ioot Mi- 



ioo=0 
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Claim. For each < ioo < 2Nn[k(oo) : k], hi^^Qi and h[ + Q £ are local systems 
onA°°. 

Proof. We prove the statement for h ioo ^Q e , and argument for the other one is the 
same. Let S = 2Np~ 1 (oo) — > A°° be the universal family of subschemes of F|,4°°. 
Then by the definition of A 00 , 5 rcd = p- x (oo) is finite etale over A°° . Let T -> A°° 
be a Galois etale cover which splits S red /A°°. We base change the situation from 
.4°° to T: 

rt -r jn[k(oo):k] a 



■A° 



where St splits into pieces S r . Each 5™ d = T and S r is the 27V th infinitesimal 
neighborhood of 5' cd C Y x^oo T. Therefore we have a Cartesian diagram 

IX LL Hiib v (s r /T) — »-u miw-is/A™) 



T ^.4°° 

where the product is the fibered product over T. By proper base change, we have 

2Nn[k(ao):k] n[fe(oo):fc] / 2N \ 

(4A6) t*h ioo ,Mt= (g) • 

icc=0 r=l \i r =0 / 

Now each finite map 5 r — + T has geometric fibers of the form Spec fl[x]/x 2N , 
therefore the reduced geometric fibers of each Hilb v (S r /T) is a single point if 
< i r < 2N, hence hf r ^Q e = Q e . By (|4.4.6|) . we conclude that t*h ix ^Q e is a 
local system on T, hence hi^^Qf is a local system on A°° because t : T — > A°° is 
etale. □ 

Since each g^~, S Qg is a middle extension ( Proposition 13 . 3 . 21 and EUT2")) and each 
h' t ^(Q>£ is a local system, t^g^f is also a middle extension by the isomorphism 
(|4.4.5[) . Recall that we have a decomposition gjf N r Q e = (BjKr> N ,d N -j, therefore 
each L* N KD N ,d N -j is also a middle extension on (A- 5 x £>^ o )°°. 

For — g?at + 2gy — 1 < j < — 2gy + 1, we know from Lemma T4.4.2I that 

Hence 

(4-4.7) L *NfD 5 N ,j,* L dN-j - l *N K D n An-] 

are both middle extensions on (A- s xS x )°°. Now we choose N such that Nn[k(oo) : 
k] > 2gy — \ (note that gy only depends on deg(£>), hence independent of TV). Then 
for any —d < i < d, we have 



-dw + 2gy — 1 < i < dN — 2gy + 1, 



which means (according to the isomorphism ()4.4.3|) ) that f^j^Ld-i h-Nn,*Qe 
appears as a direct summand of i* N fff N i ^Ld N -i, which is a middle extension by 
(|4.4.7[) . Since ftjv«,*Q^ is a local system by the above Claim, we conclude that 
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fi)o'f S *I J d-i is also a middle extension on (A- s x £> x )°°, for any — d < i < d, as 
desired. □ 

5. Proof of the local main theorem 

We prove Theorem 12.7.11 in this section. Suppose char(F) = char(fc) = p > 
max{rt, 2}. We are left with the case k'/k nonsplit, and we assume this from 

here on. We fix a collection of invariants (a°,b°) with a?, 6° € Ojj - ' which is 
strongly regular semisimple. Suppose v&\p(A a(ubo ) is even. Let 5(a°) be the local 
Serre invariant associated to the algebra R a o. i.e., 6(a ) — diim-(ii 0o /ii a o) where 
R a o is the normalization of R a o . 

5.1. Local constancy of the local moduli spaces. In this subsection, we prove 
an analogous statement to [T21 Proposition 3.5.1] in our situation. This is a geo- 
metric interpretation of the Harish- Chandra's theorem on local constancy of orbital 
integrals. 

5.1.1. Proposition. There is an integer N > 1 (depending on (a°,b )) such that for 
anyfieldQ D k and any collection of invariants (a,b) with a%, 6j € (O , E®k&') cr > 
if (a, b) = (a , 6°) mod ru N , then 

(1) (a, b) is strongly regular semisimple; 

(2) 6(a) < 6(a°)+n/2; 

(3) There are canonical isomorphisms of schemes over f2: 

Proof. We stick to the case ft = k, the general case is argued in the same way. 
First of all, by Lemma \2. 2. 41 the strong regular semisimplicity of (a, b) is checked 
by the nonvanishing of polynomials equations with O^-coefficients in a^,^: the 
discriminant Disc(P a ) of the polynomial P a (t) — t n — a\tn — 1 • • • + (— l)"a„ and 
the A-invariant A a ^- Whenever (a, b) = (a°,b°) mod or*, we have 

Disc(P Q ) = Disc(P a o) mod m N 

A a> fc = A a o b o mod vo N 

If we choose N > max{valp(Disc(P a o)), valp (A a o b o)}, then whenever (a, b) = 
(a°,b°) mod w N , Disc(P a ) and A aj6 are nonzero, hence (a,b) are strongly regular 
semisimple. 

Now fix this choice of N and any (a, b) such that a,-, bi G [Op ®fe fi) ^ -1 ) and 
(a, 6) = (a , 6°) mod ro^. Let 7 = j a o b o and 7 = 7 aj b 
By the formula for 5(a) ([4] and p72|. §3.7]), we have 

(5(a) < val F (Disc(P a ))/2 = val F (Disc(P a o))/2 < 8(a°)+n/2. 

Since N > val^(A a o b o) — vali?(A a ,;,), we have 

(5.1.1) R v a h(Ra) = (R v a /™ N R v a )h(R a /™ N R a ) 

(5.1.2) R V a oh°(R a o) = {Rlo/vj N Ry a0 )h\R a0 /vj N R a o). 

We prove that b and ■M^°^o b o are canonically isomorphic. Firstly we have 
a canonical isomorphism of Op /w N Op- algebras 

l : Rajvj^ R a — R a o/w N R a o. 
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We also have a commutative diagram 

R a /w N R a RIIvj n RI 

R a o/m N R a0 ^RV /w N RV (> 

because 7 mod w N only depends on (a,b) mod w N . Therefore, from (|5.1.1|) and 
(|5~L2f we conclude that R y a h(R a ) as an R a / m N R a -modxi\e is canonically isomor- 
phic to R a ^ / , y (R a o) as an R a ° /tz N Ra°- m °dule. Looking back into the definition 
of A4 l °a b , we observe that this scheme canonically only depends on R^/j(R a ) as 
an i? Q -modulc. Therefore we get a canonical isomorphism M l °^ b = A^'°^ b o- 
The argument for the other isomorphism A/"„°£ = A/^o^o is the same. □ 

5.2. Preparations. We fix a smooth, projective and geometrically connected curve 
X over k of genus g with a /c-point xq. Also fix an etale double cover tt : X' — > X, 
also geometrically connected, with only one closed point x' Q above xq- We choose 
identifications Op —* Ox,x and Oe — Ox', x ' - 

Fix an integer S > S(a°) + n/2. Fix effect divisors D — 2D' and Do on X, 
disjoint from xq, such that 

deg(D) > ma,x{c s ,2g + 2Nn+ 1} 

deg(A>) > ^ -deg(£>) + 2 5 + max(-,(l + i)ivl 
2 [n n ) 



These numerical assumptions will make sure that all the numerical conditions in 
the propositions or lemmas of the paper (including those which we are about to 
prove) are satisfied. 

For each closed point x : Specfc(ir) — > X, let vu x be a uniformizing parameter of 
Ox.x and F x the field of fractions of Ox,x', let Frob x be the geometric Frobenius 
element in Gal(fc(a;)/fc(a;)). Let E x be the ring of total fractions of Ox', x - Let rj x 
be the quadratic character of F* associated to the quadratic extension E x /F x . 

Recall that the double cover 7r : X' — > X gives a local system L according to 
Notation ll.6.41 Let L x = x*L be the rank one local system on Specfc(x) given by 
the pull-back of L via x : Spec k (x) — * X. Then we have 

(5.2.1) Vx (w x )=Ti(Fiob x ,L x ) 

Let Aq x Bo c A x B be the affine subspace consisting of (a, b) such that 

(a,b) = (a°,b°) mod w N 

5.2.1. Lemma. Let A' C Aq HA- s the open locus of a such that Y a is smooth away 
fromp- 1 ^). Ifdeg(D)>2g + 2Nn + l and 2 deg(D ) > 2g + N- 1, then A' and 
Bq are nonempty. 

Proof. First we have to make sure that Ao and Bq are nonempty. For this it suffices 
to show that the following evaluation maps at a 7V th infinitesimal neighborhood of 
x are surjective: 

H°(X,C(D)^) -► F /m N ,l<i<n 
H {X,Ox(2Do)®£(D)®*) -> O F /w N ,0 < i < n - 1 
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which is guaranteed as long as deg(D) > 2g + N — 1 and 2 deg(£>o) > 2g + N — 1. 

Next we make sure that Ao H.A mt is nonempty. By Proposition ^ . 1 . ll any a £ Ao 
is strongly regular semisimple at Xo, hence in particular R a is reduced. This implies 
Ao C A^. By Lemma [3XU 

coding (A® - A int ) > deg(D) > 2Nn > codirn^ („4 ) 

Therefore Ao n A int ^ 0. 

Finally we prove that A' is nonempty. We base change the whole situation to 
k. We use the argument for the Bertini's theorem. For details, we refer to [12j 
Proposition 4.6.1]. We only point out the for deg(£>) > 2g + N + 1, the evaluation 
maps at both xq and any other x <E X(k): 

ev(x ) © ev(x) : H°(X ® fe k, C(D)^) -> F /m N O x ^Jw\ 

is also surjective. This is all we need to apply the Bertini argument. 

We still have to check that for each a £ A' , 5(a) < 8. But since Y a is smooth 
away from p~ 1 (xq), we have 8(a) — 8(a,x) < 8(a°) + n/2 < 8 by Proposition 
15X17 21 . □ 

Since A' is geometrically irreducible (because it is an open subset of an afhnc 
space) and nonempty, it contains a fc m -point for every m > niQ. Now we fix m > tuq 
and fix a point a e A'o(k m ). We base change the whole situation from k to k m ; 
in particular, let X m — X g)/. k m . Since D and Do are disjoint from xq, the 
trivializations that we fixed allow us to get er-equivariant isomorphisms: 

R a (0 E ®kk m ) C Y v(-L'o)|spocO > ,,, xo ; 
R y a (0 E ®kk m ) = w Ya 7x(- D o)|specO^ (a!0 - 

5.2.2. Lemma. Ifn(deg(Do) — g + 1) > (n + l)N + gy, then there exists b £ Bo (k m ) 
such that for each closed point x ^ xo of X m , f, x (fc m (ir)) ^ 0. 

Proof. We first choose any 6 G Bo(k m ) (which exists because Bq ^ is an affine 
space over k). The pair (a, 6) determines 7 = ~f a ,b ■ Ra ®k k m <^-» i?„ ®fc fc m . 

Claim. There exists T 1 £ 7'ic(y a ') and a homomorphism h : J-' a* T N such that 

(1) a*h v = h; 

(2) h is an isomorphism away from p'~ x (x' ) ; 

(3) There is an isomorphism 1 : J~'\spccO Y i XQ — Ra(0 E ®fc k m ) such that the 
following diagram is commutative 

J~ |SpecO y , )a . s- cr J- |Spec0 y /,* o 

i?a(C B ®fe fc m ) i?^(0 B ®fe km) 

Proof. Let i? be the normalization of R a ®kkm- We can write R — R t where each 
Rt is a DVR with residue field k m (Rt). After choosing an isomorphism R^ ®fc fcm — 
-Ra ®fc fcm of i? a ®fc fc m -modules (which exists because R a (S>k k m is Gorenstein), we 
can view 7 as an element of R a <8>fc k m . In terms of the above decomposition, 
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7 = (71 ) ' ' ' i Tr) where jt € Rt, and we have 

r 

(5.2.2) valF(A Q o b o) = val F8fcfem (A Q:b ) = val Rt (■jt)[k m {Rt) ■ k m ]. 

Since vali?(A a o b o) is even, the RHS is also even. 

Then condition (1) is automatically satisfied once we have such an h since Y£ 
is geometrically irreducible. Let U' = Y' a — p'^ 1 {x' a ). Suppose J-' is a line bundle 
on U' with an isomorphism hg : J-q ^ o* T§ . Such a line bundle exists because 
we have a Kostant section A int -> J\f mt \ A int by [10, §2.3] when D = 2D'. The 
existence of the Kostant section requires that char(/c) > n. Any other cr-conjugate 
self-dual line bundle on V has the form T' <£> Q where Q is a line bundle on V 
with an isomorphism s : Q — \ a*Q~ l . We want to glue (T' G,ho <3 s) with 
(Ro,{Oe <Z>k km),crR 7) over the punctured formal neighborhood U' XQ of p' ( ~' 1 {x' ) 
to get the desired pair (J 7 ', h). For this we have to choose an isomorphism 

p: H®Q\u'„ =R a {E® k k m ) 
such that the following diagram is commutative: 



R a {E® k k m ) 



(T07 



Ra(E 1 



We can translate this problem into the language of ideles. Let K (resp. K') be 
the function field of the geometrically connected curve Y a (resp. Y a ') over k m . Let 
Ak (resp. Ak') be the set of adeles of K (resp. K') with (resp. Ok 1 ) the 
product of local rings integers. Let Nm : A£, — > A^- be the norm map. Then R 
is the product of local fields corresponding to places of K over xq. Thus we get a 
canonical embedding R C Aj^ ■ In particular, we can identify 7 with an idele in A^ 
which is nontrivial only at places over xq. 

A choice of the above pair {Q, s, p) (up to isomorphism) is the same as the choice 
of an idele class 6 e K' x \A^,/0^, such that Nm(0) = 9 ■ o9 = 7 as an idele class 



in K x \A x /O x . Let W K > and W K be the Weil groups of K' and K. By class field 
theory, we have the following commutative diagram 



K' 



K' 



Art t 



Art_p 



■1/2 



Gal(K'/K) 



where the map d is defined by 

9(0 = X! val K v (Zv)[h ■■ k m ] mod 2 

V nonsplit 

where v runs over all places of K which is nonsplit in K' and k v is the residue field 
of Ok v - Now to solve our problem, i.e., to solve the equation Nm(0) = 7, the only 
obstruction is d(-f). Since xq is nonsplit in X' m , a place v over xq is nonsplit in K' 
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if and only if [k v : k m ] is odd. Therefore 

9(7) = X! valjf v (7„)[fc w : fc m ] 

d|ieo :&tti] odd 

= valjc t ,(7^)[fc^ : fc m ] 

r 

= y^valji t (7t)[fc(-R f ) : fc m ] mod 2. 
t=i 

But the RHS is even by (|5.2.2p . hence 9(7) = 0. Therefore we can always find 9 € 
K' X \A^-, such that Nm(6') = 7. Translating back into geometry, we have found the 
desired (Q, s, p) and hence we can glue (J-q Cg) G, ho (Sis) with (R a (C>E <8>fc k m ), ^07) 
over U' XQ to get the desired (J 7 ', h). The isomorphism t : .F'lspccCV/ x — Ro,{Oe ®fc 
fc TO ) is tautologically given by the gluing. □ 

Now we pick such a triple (T' , /i, t) from the above Claim. By construction, we 
have 

X (r a ',^') = -2n(g- 1) -val F0fcfcm (A a , fc )/2 
Claim. There is a homomorphism a' : Oy'{—Dq) — > J 7 ' suc/i £/iai i/ie composition 

lo a' : R a {0 E ®k k m ) = O Y ^x F'\s P cco Y ,^ o R a (0 E ®fc k m ) 
is the identity modulo w N . 

Proof. Consider the following evaluation map at the N th infinitesimal neighborhood 
ofp'- l (x' )cY>: 

ev : J* (Do) -> ^'(A))|s P ccev,, X0 ^ R a {0 E ® fe k m ) ®o F (O f /w n ). 

Let /C be the kernel of cv, which is a coherent sheaf on Y' a . By Grothendieck-Serre 
duality, we have 

ff 1 (^,/C)SHam y2 (/C,wy,) v . 

But since 

x(F ',/C) = x(F a ',^'(A)))-27iiV 

= - val F8fcfem (A a , fc )/2 + 2n(deg(D ) - 9 + 1) - 2?iiV 

> 2n(deg(D ) - fl + 1) - (2n + 1)N 

> 2g Y > g' Y - 1 = x(Y^,uy>), 

we must have Hom(/C, cjy) = 0, therefore H 1 (Y l ! t ,IC) — 0. This implies that 

ev : Homy^eV^-A)),^') - H Q (Y^F'(D )) -» i? a (0£ ®fc M ®o F (Of/w*) 
is surjective. Hence there exists a' E Homy (0yv(— Dq),F') such that ioev(a') = 1 
mod tzj^. □ 

Now let 7' be the composition 

O Y ,(-D ) ^ T' i a*.F' v wj^Do). 

Then 7' gives back another b' S B(k m )- From the construction it is clear that 
b' = b mod tn^, therefore 6' G 2?o(fc m ). Now for each closed point x ^ xq of 
X m , the local moduli space Af* b , (k m (xj) is nonempty because it contains a point 
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given by J-'\spccO Y , x (this is self-dual because h is an isomorphism over x ^ xo by 
construction). Therefore, the pair (a, b') satisfies the requirement of the Lemma. □ 

5.3. The proof. Now for each m > m , we have a pair (a,b) € A' (k m ) x Bo(k m ) 
such that the condition in Lemma 15.2.21 holds. Using Theorem 14.4. 1[ taking the 
stalks of the two complexes in (|4.4.1|) at the point (a, 6), we get an isomorphism of 
graded Frob™ modules: 



(5.3.1) H'(M ha ,b ®k m k, L d ^) = H'(Af a ,b ®k m k, Q t ). 

i— — d 

By the product formulae (|3.4.1[) and (|3.4.2[) , we can rewrite (|5.3. 1|) as 

® ( H ' (( Res ^(-)A m M La*,bJ ®t m k, Re Skm(x)/km Lf») j 

x£\X m \ \ i x ) 

<* (g) H' ((R£S km(x y km Nl, b J ® km kMe) ■ 

xe\x m \ 

Here Res km M/ km L x is the local system of rank one on Res km ( x y km Speck m (x) 
induced from L x . We have such a tensor product decomposition on the LHS because 
the local system Ld-i on Mi t a,b, when pulled-back via the isomorphism (|3.4.ip . 
becomes ^ xe \ Xm \ Res km{x)/km L® 1 * on rLe|x m | Res k m (x)/k m Mf x Qx bx (cf. Lemma 

MM- 

We use the following abbreviations 

Ml := 0IF ((Res fcm(x)/fem j0 . j6 J ® km k,Res km(x)/km L«*») 

JV* := ^'((Res fcm(:c)/fcm AC x , 6x )®fc m fc,Q£). 

val F (A a o b o) 

M i := ^(Ai^o, 6 o®fcfe,^(^ /fc )® i ); 

»=0 

JVg := ^(^>® fe fc,^). 

where the first two are Frob™-modules and the last two are Frobfc-modules. 
By Proposition 15 . 1 . ll we have 

M j — M j ■ N j — N j 

lvl x — ul o I ly x — iv o 

as Frob™-modules. 

On the other hand, since we have assumed that Y a is smooth away fromp < ^ 1 (cco), 
for any x ^ x , the local moduli spaces M.* flx bx and Af* b are zero-dimensional, 
hence no higher cohomology. For x ^ xo, we write M x (resp. N x ) for (resp. 
N x ). Therefore for each j, we get an isomorphism of Frob™-modules: 

M$®1 (g) M x J S N> <g> (g) 

\a;o/a:G|A m | / \x #xe|X m | 
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Taking the traces of Frob™, and using the Lefschetz trace formula for M.\ 
and ^ , we get for any j > 0, 

(5.3.2) Tr(Frob£\M(*) J] fer h (w ai ) i >#MZ, aBtba (k m (x)) 

x ^xe\X m \ \ i x 

(5.3.3) = Tr(Frob fc m ,^) J[ #M^(k m (x)). 

x ^£xe\x m \ 

Here we used (|5.2.1[) . Since Y a is smooth away from p~ 1 (xo), for x ^ xq, R a:c — 
Oy a ,x is a product of DVRs, and we can apply Lemma [2T5T6] to conclude that 

^r] x {w x y^M^ axh {k m {x)) = #AfZ xjK (k m (x)),Vx + x . 

Moreover, the RHS for each x ^ xq is nonzero, because (a, b) satisfies the condition 
in Lemma T5. 2. 21 Therefore (|5.3.2p implies 

Tr(Frobr,Mi5') = Tr(Frob£\ i\tf),Vj > 0. 

Since this is true for any m > mo, we get an isomorphism of semi-simplified 
Frobfc-modules 

Mi' ss = Nfr ss . 

But by Lemma 12.7.31 Mq and A^g are isomorphic as Frob^-modules. Therefore 
we can conclude that Mq = Nq as Frobfc-modules since the unipotent part of the 
Frobfc action is uniquely determined by that of Frob^ by taking the square root. 
This proves the main theorem. 
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